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STATEMENT  OF  VOEK 

As  the  fuel  situation  worsens,  the  Air  Force  will  need  to  maximize 
the  effectiveness  of  computer-based  synthetic  flight  training.  The 
training  of  motor  skills  in  synthetic  trainers  is  a  critical  element  of 
flight  training  because  of  the  new  tasks  pilots  may  be  called  upon  to 
perform  in  future  warfare.  Such  tasks  might  include  low-level  fast 
bomber  weapons  presentation,  high-speed,  accurate  firing  of  air-to-air 
and  air-to-ground  missiles,  evasion  of  enemy  missiles,  and  formation 
flying.  By  optimizing  the  use  of  synthetic  trainers  for  the  original 
learning,  retention,  and  transfer  of  these  critical  motor  skills.  Air 
Force  personnel  should  be  better  equipped  to  perform  these  all-important 
tasks. 

A  major  difficulty  in  motor  skills  training  is  the  large  intersubject 
variability  resulting  when  only  one  fixed  training  procedure  is  employed. 
However,  modern  computer  technology  provides  the  capability  to  process 
large  amounts  of  continually  varying  data  such  that  the  instructional 
environment  could  be  adapted  to  the  learning  characteristics  of  the 
individual  student.  By  using  the  computer  inherent  in  the  design  of 
synthetic  flight  trainers,  the  development  of  effective  ground-based 
training  programs  for  individual  students  in  pilot  training  should  be 
possible. 

Individualized  training  involves  adapting  instructional  practices 
to  individual  requirements  either  by  optimizing  the  instruction  sequence 
or  by  selecting  appropriate  training  alternatives  for  individual  students. 
Traditionally  motor  skills  training  has  employed  a  fixed-difficulty  strategy 
in  which  students  are  immediately  exposed  to  the  criterion  level  of  task 
difficulty  and  student  error  decreases  as  training  progresses.  Unfortunately, 
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this  strategy  has  no  provision  for  individual  differences  in  prior  experience, 
rate  of  learning,  or  learning  style.  Therefore,  the  training  task  may  be 
too  easy  at  times  for  some  individuals  and  too  difficult  at  times  for 
others,  resulting  in  an  Inefficient  use  of  training  time. 

Two  general  approaches  to  individualizing  motor  skills  training  are 
possible  by  considering  micromodels  and  macromodels.  The  micromodel 
approach  assumes  that  each  student  follows  his  or  her  unique  learning  model 
through  training.  One  example  is  adaptive  motor  skills  training.  In 
adaptive  motor  skills  training  the  difficulty  or  complexity  of  the  training 
task  varies  directly  as  a  function  of  student  performance.  If  the  student's 
performance  is  within  a  specific  error  tolerance,  the  task  difficulty  or  com¬ 
plexity  increases  until  an  exit  criterion  is  reached.  If,  on  the  other  hand, 
the  trainee  is  outside  a  specified  error  tolerance,  the  task  difficulty  is 
decreased.  Kelley  (1969)  sunmarized  an  adaptive  training  system  as  re¬ 
quiring  a  continuous  measure  of  trainee  performance,  one  or  more  adaptive 
variables  that  can  change  the  task  difficulty  or  complexity,  and  a  logic 
system  for  automatically  changing  the  adaptive  variable(s) .  The  usual 
approach  taken  in  adaptive  procedures  is  to  use  a  single  logic  system. 

Even  though  this  one  logic  system  provides  a  variety  of  individual  task 
difficulty  profiles,  it  may  not  provide  enough  flexibility  for  various 
styles  of  information  processing. 

The  macromodel  approach  to  Individualizing  training  assumes  that  only 
a  limited  number  of  learner  types  exist.  Students  are  categorized  on 
various  dimensions  as  to  learner  type  and  assigned  to  the  optimal  training 
alternative.  The  macromodel  approach  has  been  pursued  by  Cronbach  and 
his  colleagues  (Cronbach  and  Snow,  1977)  with  only  limited  success.  The 
difficulty  arises  from  the  inability  to  specify  aptitude- treatment  inter¬ 
actions  and  to  associate  these  interactions  with  specific  types  of  training. 
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An  alternative  macrooodel  approach  avoids  the  need  to  specify  underlying 
aptitude-treatnent  interactions  by  using  regression  equations  to  predict 
individual  performance  for  each  available  training  type.  A  comparison  of 
predicted  performance  under  each  training  type  determines  the  optimal 
training  condition  for  the  student.  No  categorizations  of  student  learning 
styles  or  training  alternatives  are  necessary. 

The  goal  of  this  programmatic  research  effort  was  to  investigate  the 
ramifications  of  both  micromodels  and  macromodels  in  individualizing  motor 
skills  training.  Specifically,  during  this  year's  effort,  additional 
research  was  conducted  on  the  use  of  feedback  in  adaptive  training,  the 
extension  of  regression  approaches  to  training  group  assignment,  and  the 
evaluation  of  subject  population  differences. 

Adaptive  Training:  A  Micromodel 

One  important  issue  in  adaptive  training  deals  with  the  role  of 
augmented  feedback.  Bilodeau  and  Bilodeau  (1961)  state  that  "studies  of 
feedback. . .show  it  to  be  strongest,  most  important  variable  controlling 
performance  and  learning."  However,  the  Kelley  (1969)  adaptive  logic 
system  minimizes  the  usefulness  of  intrinsic  task  feedback.  By  manipulating 
task  difficulty  based  on  performance,  relatively  constant  level  of  error 
is  maintained  over  time.  Consequently,  the  student  sees  no  progress  in 
terms  of  error  and  may  need  augmented  feedback  in  terms  of  the  level  of 
task  difficulty.  Although  he  never  evaluated  his  position  experimentally, 
Kelley  contends  that  augmented  feedback  in  terms  of  task  difficulty  is 
essential  in  adaptive  training. 

Norman  (1973)  compared  the  use  of  a  task  difficulty  meter  with  the 
use  of  no  feedback  in  adaptive  training  and  found  some  savings  in  learning 
a  transfer  task  when  the  task  difficulty  meter  was  used.  However,  the  two 
experimental  conditions  compared  were  from  two  separate  studies  and  involved 


different  subject  pools.  In  Experiment  I  (feedback  condition)  college 
students  and  recruits  from  the  Navy  Training  Center  were  involved.  In 
Experiment  II  (no  feedback  condition)  only  college  students  were  used  as 
subjects.  This  methodological  problem  permits  only  tentative  conclusions 
to  be  drawn  from  the  results. 

A  more  recent  study  by  Cote,  Williges,  and  Williges  (1978)  indicated 
that  augmented  feedback  during  training  in  terms  of  task  difficulty  and 
accuracy  of  performance  had  no  effect  on  learning  a  two-dimensional  pursuit 
tracking  task  using  either  a  fixed  difficulty  or  an  adaptive  training 
strategy.  Certainly  if  subjects  used  the  visual  feedback,  an  increase  in 
visual  workload  occurred.  Because  the  visual  load  In  the  training  task  was 
inherently  high,  it  is  unclear  whether  visual  feedback  was  unnecessary  or 
unused. 

High  visual  workload  is  common  with  many  motor  skills,  such  as 
controlling  an  aircraft  or  an  automobile.  Logically  feedback  might  be 
more  useful  if  provided  through  a  channel  other  than  visual.  For  example, 
Gilson  and  Ventola  (1976)  have  successfully  used  tactual  augmented  feedback 
to  present  information  concerning  the  flight  path  to  pilots  during  approach 
and  landing  operations. 

However,  vision  appears  to  be  the  dominant  modality  in  motor  learning. 
Disagreement  exists  among  motor  learning  theorists  concerning  the  value 
of  feedback  in  other  sense  modalities.  The  current  research  explored  the 
usefulness  of  auditory  augmented  feedback  in  adaptive  motor  skills  training. 
Specifically,  the  effectiveness  of  visual,  auditory,  auditory  and  visual, 
and  no  augmented  feedback  were  compared. 

Training  Assignment;  A  Macromodel 

Using  cognitive  tasks,  researchers,  such  as  Pask  (1976),  have  demonstrated 


that  when  the  student's  preferred  learning  style  and  the  teaching  strategy 
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employed  are  mismatched  learning  is  severely  disrupted  in  terms  of  compre¬ 
hension  and  retention.  Instructional  theorists  have  also  noted  the  importance 
of  learning  style.  Carroll's  (1963)  theory  suggests  that  the  degree  of 
learning  a  given  task  is  a  function  of  the  amount  of  time  spent  learning  the 
task  in  relation  to  the  amount  of  time  needed  to  learn  the  task.  Time  needed 
is  based  upon  learning  under  optimal  conditions  where  optimal  conditions 
are  defined  by  the  student's  learning  style  preference.  Bloom  (1976)  provides 
three  predictors  of  time  to  learn:  (1)  cognitive  entry  behaviors  (prior 
experience  with  the  task),  (2)  affective  entry  behaviors  (motivation  level 
of  the  student),  and  (3)  quality  of  instruction  (appropriateness  of  the 
training  situation  for  the  student) . 

To  determine  the  appropriateness  of  the  training  situation  for  a 
specific  student  a  model,  such  as  a  regression  equation,  may  be  used. 
(Kaskowitz  and  Suppes,  1978,  have  suggested  that  a  regression  equation  may 
be  considered  to  be  a  mathematical  model  in  the  sense  that  a  linear 
relationship  between  time  to  train  and  certain  independent  variables  is 
hypothesized.)  Using  regression  equations  to  predict  time  to  complete  a 
course  on  stock  control  and  accounting,  Wagner,  Behringer,  and  Pattie 
(1973)  found  that  grouping  students  according  to  mode  of  instruction 
(audio-visual  or  programmed  instructions)  improved  prediction.  The  improve¬ 
ment  In  prediction  suggests  that  training  type  interacted  with  some  individual 
characteristic  of  the  students.  It  follows  that  a  comparison  of  predicted 
scores  associated  with  the  various  training  types  might  yield  an  optimal 
training  assignment. 

A  preliminary  evaluation  of  the  use  of  multiple  regression  for  training 
group  assignment  has  been  conducted  using  the  Air  Force  Advanced  Instructional 
System's  Inventory  Management  course.  McCombs  (1979)  reports  modest  savings 
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in  training  time  when  regression  models  were  used  to  select  students  for 
alternative  training  modules.  However,  because  the  study  was  conducted 
within  the  constraints  of  an  operational  training  system,  several  limitations 
should  be  noted.  First,  alternative  treatments  were  available  only  in 
selected  lessons  (27%  of  the  course).  Second,  no  students  were  purpose¬ 
fully  mismatched,  so  the  discrlminability  of  the  selection  procedure  could 
not  be  tested.  Third,  selection  of  the  optimal  training  type  could  be 
overridden  when  the  instructional  materials  were  not  available  or  when  an 
instructor  changed  a  student's  assignment.  However,  even  with  these 
limitations,  consistent  savings  in  time  to  learn  a  cognitive  task  were 
reported  when  regression  modeling  was  used. 

The  current  research  at  VPI  extends  the  regression  model  approach  of 
training  group  assignment  to  the  perceptual-motor  learning  domain.  Specifi¬ 
cally,  each  student  was  assigned  to  a  training  strategy  on  the  basis  of 
predicted  scores  from  baseline  regression  models  of  training  time-to-exit. 
Students  were  matched  (shorter  predicted  time),  mismatched  (longer  predicted 
time),  or  randomly  assigned  to  fixed-difficulty  or  adaptive  training  to  learn 
a  two-dimensional  pursuit  tracking  task. 

Subject  population  validation.  One  critical  element  in  any  research  is  the 
selection  of  an  appropriate  subject  population.  The  current  set  of  regression 
models  for  training  group  assignment  has  been  developed  using  civilian 
university  students  as  subjects.  Both  men  and  women  were  involved.  These 
students  are  in  the  same  age  range  and  probably  possess  many  characteristics 
similar  to  young  military  officers.  However,  a  comparison  of  the  performance 
of  this  civilian  population  with  that  of  an  appropriate  military  population 
would  be  useful  to  establish  the  validity  of  the  regression  models  and  the 
regression  approach  to  individualized  instruction  in  flight  training.  In 


addition,  a  comparison  of  any  differences  in  the  reliable  predictors  for 
males  and  females  would  be  useful. 

With  these  goals  a  joint  research  project  between  the  Human  Factors 
Laboratory,  Virginia  Polytechnic  Institute  and  State  University,  and  the 
Department  of  Behavioral  Science  and  Leadership,  U.S.  Air  Force  Academy, 
was  conducted.  A  series  of  pretests,  Including  the  tests  used  to  generate 
the  original  regression  models,  were  given  to  a  set  of  VPI  students  and 
a  set  of  Air  Force  Academy  cadets.  Scores  on  these  tests  were  used  to 
generate  new  multiple  regression  prediction  equations  predicting  post¬ 
training  performance  on  a  desk-top  flight  trainer. 
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STATUS  OF  THE  RESEARCH 

Research  Tasks 

Two  general  tasks  were  used  In  the  research  studies  completed  in 
the  1978-79  contract  year.  The  first  task  is  a  two-dimensional  pursuit 
tracking  task  generated  by  a  PDF  11/10  digital  computer  interfaced  with  a 
Tektronix  4014-1  cathode  ray  tube  display  and  a  Measurement  Systems  Model 
435  isometric  control  stick.  The  second  type  of  task  uses  an  ATC-610 
desk- top  flight  trainer. 

In  the  tracking  studies  each  student  completes  a  series  of  3-minute 
tracking  trials  to  learn  a  two-dimensional  task  in  which  random  functions 
are  used  to  determine  the  coordinates  of  the  forcing  function  symbol  (X) 
on  the  display.  The  control  output  (0)  is  generated  using  inputs  from  the 
analog  controller.  Task  difficulty  can  be  manipulated  automatically  by 
the  computer  using  a  linear  optimization  model,  maintained  at  a  fixed  level 
of  difficulty,  or  controlled  by  the  student.  Integrated  absolute  tracking 
error  is  recorded  automatically  by  the  computer.  Figure  1  shows  the  version 
of  this  task  where  dynamic,  augmented  feedback  in  terms  of  task  difficulty 
and  tracking  accuracy  is  presented  in  addition  to  the  forcing  function  and 
controller  symbols.  Exit  criterion  is  obtained  when  the  student  maintains 
exit  criterion  task  difficulty  and  acceptable  accuracy  for  a  specified 
period  of  time  (usually  20  seconds).  Following  training  and  a  short  rest 
period,  each  subject  completes  a  transfer  task  in  which  no  augmented  feed¬ 
back  is  provided,  and  task  difficulty  is  changed  periodically.  Three  levels 
of  difficulty  are  used:  the  same  as  exit  criterion  in  training,  more 
difficult  than  exit  criterion,  and  less  difficult  than  exit  criterion.  All 
software  for  the  tracking  task  was  written  by  John  E.  Evans,  III.  (Figure  2 
illustrates  the  software  configuration  for  the  tracking  task.) 
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Figure  2.  Software  configuration  for  the  two- 
dimensional  pursuit  tracking  task 
developed  by  John  E.  Evans,  III. 
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The  flight  task  used  In  the  population  comparison  research  Involved  a  desk¬ 
top  trainer  equipped  with  the  flight  instruments  normally  associated  with 
light  aircraft.  Figure  3  illustrates  the  instrument  panel  of  this  trainer. 
Students  received  approximately  40  minutes  of  training  via  audio  cassette 
and  color  photographs  on  basic  flight  controls  and  instruments.  Two  practice 
trials  on  each  manuever  were  given.  During  the  first  practice  trial  the 
instructor  could  intervene  if  the  student  did  not  appear  to  understand  the 
manuever.  Students  were  tested  twice  on  each  of  four  manuevers,  once  in 
smooth  air  and  once  in  rough  air.  Simulated  turbulence  was  introduced 
through  an  electrical  signal  which  was  reflected  in  the  aircraft  Instruments. 
The  turbulence  control  is  a  six-position  switch  labeled  OFF  and  1-5.  The 
switch  was  OFF  for  smooth  air  conditions  and  set  at  position  2  for  rough-air 
conditions.  The/fasks  in  the  trainer  were  climb  on  a  heading  at  the 
specified  vertical  velocity,  cruise  straight  and  level,  descend  at  a 
specified  vertical  velocity  on  a  given  heading,  and  make  a  level  standard 

rat^^urn  of  180  degrees.  The  trainer  was  trimmed  for  level  flight,  and 

/ 

students  were  not  required  to  control  airspeed,  fuel  mixture,  or  manifold 
pressure.  As  a  result  the  task  was  quite  similar  to  a  multidimensional 
cracking  task. 

Augmented  Feedback  In  Adaptive  Training 

To  examine  the  effects  of  various  forms  of  augmented  feedback  96  male 
subjects  were  taught  a  two-dimensional  pursuit  tracking  task  using  either  a 
fixed-difficulty  or  adaptive  training  procedure.  Subjects  in  each  training 
procedure  were  placed  in  one  of  four  feedback  conditions:  (1)  auditory 
off-course  feedback  only,  (2)  visual  off-course  feedback  only,  (3)  auditory 
and  visual  off-course  feedback,  or  (4)  no  augmented  feedback.  Subjects  trained 
adaptively  also  received  feedback  in  terms  of  task  difficulty  changes.  All 
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Figure  3.  ATC-610  personal  flight  trainer  instrument  panel. 
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subjects  received  a  6-minute,  no-feedback  transfer  task  similar  to  the 
transfer  task  used  in  previous  studies.  Three  levels  of  task  difficulty 
were  presented,  and  the  order  of  presentation  of  the  three  levels  of  task 
difficulty  was  counterbalanced  across  subjects.  The  complete  experimental 
design  is  given  in  Figure  4. 

Visual  feedback  was  provided  by  bar  graphs  located  on  either  side  of 
the  tracking  area.  Auditory  feedback  was  provided  through  a  loud  speaker. 

Two  distinct  tones  varying  from  a  continuous  tone  to  silence  (criterion  level) 
were  used  to  present  task  difficulty  and  tracking  accuracy  Information. 

No  reliable  effect  due  to  feedback  condition  or  training  procedure 
was  reflected  in  training  time-to-exit  data.  However,  subjects  trained 
adaptively  performed  significantly  better  (£*.025)  in  transfer  than  those 
subjects  trained  in  the  fixed-difficulty  situation.  This  may  have  been 
partially  the  result  of  students  trained  adaptively  having  received 
practice  at  various  levels  of  task  difficulty  during  training. 

These  results,  as  well  as  the  results  of  the  previous  research 
(Cote,  Williges,  Williges,  1978),  suggest  that  augmented  feedback  does 
not  enhance  training  in  a  closed-loop  adaptive  training  system  with  clearly 
discernable  intrinsic  task  feedback.  However,  task-difficulty  feedback  may 
be  useful  to  maintain  a  steady  increase  in  performance  over  an  extended  time 
period,  and  off-track  feedback  may  be  effective  in  enhancing  performance  in 
adaptive  systems  lacking  clearly  discernible  intrinsic  feedback. 

The  results  of  this  study  on  visual  and  auditory  feedback  in  adaptive 
skill  training  are  summarized  in  the  following  report: 

Cote,  D.  0.  The  combined  effect  of  various  types  of  augmented 

feedback  and  two  training  procedures  on  motor  skill  learning. 

Masters  thesis.  Virginia  Polytechnic  Institute  and  State  University, 

March,  1979. 
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To  study  the  efficacy  of  the  regression  approach  to  training  group 
assignment,  multiple  regression  equations  were  used  to  assign  120  students 
to  either  fixed-difficulty  or  adaptive  training  strategies  for  learning 
the  two-dimensional  pursuit  tracking  task.  Students  were  either  matched 
(shorter  predicted  training  time),  mismatched  (longer  predicted  training 
time),  or  randomly  assigned  to  a  training  type.  An  equal  number  of  male 
and  female  students  were  used  in  each  assignment  procedure  by  training  type 
combination.  Figure  5  depicts  the  factorial  design  used  in  this  study. 

Previously  a  double  cross-validation  procedure  had  been  used  to  validate 
the  regression  equations  predicting  time  to  learn  the  tracking  task  because 
the  coefficients  of  multiple  determination  were  consistently  high,  the  two 
samples  were  combined  and  new  equations  were  generated.  These  combined 
sample  equations  were  employed  in  the  present  study  for  training  group 
assignment  (see  Table  1) .  For  details  on  the  pretest  battery  and  validation 
procedures  see  Savage,  Williges,  and  Williges  (1978) . 

Results  of  an  analysis  of  variance  on  actual  training  time-to-exit 
scores  revealed  reliable  main  effects  of  assignment  F(2,108)*17.27,£<.0001 
and  sex,  ^(1,108)*40.57,£<.0001.  Matched  subjects  required  significantly 
less  time  to  exit  than  either  random  or  mismatched  subjects,  and  males 
required  significantly  less  training  than  females.  There  was  no  reliable 
difference  between  training  alternatives  (jj*«.246). 

Use  of  the  regression  equations  to  predict  optimal  training  type  resulted 
in  savings  of  47%  of  training  time  over  random  assignment  and  53%  over  mis¬ 
matched  assignment.  Variance  in  training  time  was  reduced  approximately  40% 
by  optimizing  training  group  assignment.  Table  2  summarizes  the  reliable 
effects  from  the  analysis  of  training  time. 


MATCHED 
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TABLE  1 

Combined  Sample,  Raw  Score  Regression  Equations  for  Time  to  Learn  a 
Two-Dimensional  Pursuit  Tracking  Task  Using  Fixed-Difficulty  or 
Adaptive  Training 


Adaptive  Training 


TEa  -  2641.65  +  1.72  EF  -  256.90  MM  +  516.42  SX 
n  *  51 
R2  -  .756 

»s  ■  -740 


EF  »  Embedded  Figures  Test  (Witkin,  Oltman,  and  Raskin,  1971) 

IP  ■  Identical  Pictures  Test 

CC  -  Cube  Comparison  Test  (Ekstrom,  French,  Harman,  and  Dermen,  1976) 

MM  «  Map  Memory  Test 
SX  *  Sex  of  Student 
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TABLE  2 

Summary  Statistics  for 

Group  Selection  Study 

Effect 

Training  Type 

Fixed-Difficulty 

Adaptive 

Assignment  Procedure 
Matched 
Random 
Mismatched 

Sex  of  Student 
Male 
Female 


Training  Time-to-Exit 

X 


15.6 

17.5 


10.0 

18.7 

21.1 


11.4 

21.8 


(min)  in  the  Training 


10.1 

12.4 


7.0 

11.6 

11.7 


7.1 

12.4 


These  data  strongly  support  the  use  of  regression  equations  to  optimize 


training  group  assignment.  The  optimization  procedure  resulted  in  savings 
in  training  time  and  a  reduction  in  variance  among  students.  Interestingly, 
no  overall  difference  in  training  time  between  fixed-difficulty  and  adaptive 
training  was  noted.  If  the  study  had  employed  only  random  assignment,  one 
might  have  erroneously  concluded  that  no  advantage  is  to  be  gained  in 
providing  alternative  training  conditions. 

To  facilitate  implementation  in  operational  training  systems,  research 
is  warranted  to  examine  regression  optimization  with  additional  types  of 
predictors,  training  procedures,  and  more  complex  training  tasks.  Guide¬ 
lines  for  selection  of  viable  predictors  are  particularly  critical.  The 
experience  with  the  Advanced  Instructional  System  vividly  portrays  the 
complexity  of  implementing  innovative  training  programs.  However,  if  the 
computer  is  ever  to  provide  real  value  to  training  systems,  the  challenge 
of  the  operational  training  system  must  be  conquered. 

This  research  is  summarized  in  the  following  papers: 

Savage,  R.  E.  A  multiple- regression  information  processing  approach 
for  assigning  individuals  to  a  training  strategy.  Masters  thesis. 
Virginia  Polytechnic  Institute  and  State  University,  February,  1979. 

Williges,  B.  H.,  Williges,  R.  C.,  and  Savage,  R>  E.  Predicting 
optimal  training  group  assignment.  Proceedings  of  the  Human 
Factors  Society  23rd  annual  meeting.  Boston,  Massachusetts, 

October,  1979,  295-299. 

Joint  Air  Force  Academy/VPI  Research 

The  second  research  project  during  1978-79  involving  regression  pro¬ 
cedures  was  a  joint  effort  with  the  U.S.  Air  Force  Academy,  Department  of 
Behavioral  Sciences  and  Leadership,  and  was  coordinated  by  Lt.  Col.  Jefferson 
M.  Koonce  at  the  Air  Force  Academy  and  by  Prof.  Robert  C.  Williges  at 


Virginia  Polytechnic  Institute.  The  purpose  of  the  research  was  to  compare 
the  equations  of  civilian  university  students  with  Air  Force  cadets  and  of 
male  students  with  female  students  to  predict  performance  on  a  real-world 
flight  task.  Indirectly,  these  comparisons  might  have  implications  for 
pilot  selection  and  attrition. 

All  subjects  were  tested  on  the  pretest  battery  developed  at  VPI  which 
included  the  pursuit  rotor  and  five  information  processing  tasks.  (See 
Savage,  1979,  for  a  complete  description  of  each  test.)  The  information 
processing  tests  used  were  comparable  to  the  tests  on  the  Air  Force  Officer 
Qualifying  Test  (AFOQT)  measuring  perceptual  characteristics.  In  addition, 
the  two  tests  on  the  Psychomotor  Test  Device,  Model  1017  (PTD)  designed  by 
the  Systems  Research  Laboratory,  Dayton,  Ohio,  were  administered.  These 
tests  are  currently  under  consideration  by  the  Air  Force  as  pilot  selection 
devices  which  feature  automatic  testing  and  scoring  procedures.  The  first 
test  on  the  PTD  is  a  two-hand  coordination,  pursuit  tracking  task.  The 
display  for  Task  1  is  given  in  Figure  6.  The  second  test  involved  both  a 
two-dimensional  compensatory  tracking  task  controlled  by  a  dual-axis 
joystick  and  a  one-dimensional  compensatory  tracking  task  controlled  by 
foot  pedals.  The  display  for  PTD  Test  2  is  given  in  Figure  7.  Pretesting 
required  two  50-minute  sessions. 

Scores  from  the  pretests  plus  sex  of  student  and  institution  were  used 

to  predict  performance  on  four  flight  tasks  performed  on  the  ATC  desk-top 
flight  trainer.  Details  on  the  flight  training  and  testing  session  are  given 
previously.  The  training- testing  nackage  was  developed  by  Beverly  H.  Willige 
and  was  largely  based  upon  the  Automated  Pilot  Aptitude  Measurement  System 
(APAMS)  developed  by  McDonnell  Douglas  Corporation  for  the  Air  Force 
Human  Resources  Laboratory. 
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Figure  6.  Display  for  two-hand  coordination 
task  on  the  PTD.  (Test  1). 


22 


X-AXIS  ♦ 


•  Y-AXIS 


•  •  • 


Z-AXIS 


Figure  7.  Test  2  on  the  PTD  including  the  two-dimensional 
compensatory  tracking  task  controlled  by  a  two- 
axis  joystick  and  the  one-dimensional  compensatory 
tracking  task  (z-axis)  controlled  by  foot  pedals. 


Scoring  on  the  flight  test  was  facilitated  by  an  objective  pilot 
performance  rating  booklet  developed  by  Lt.  Col.  Koonce.  A  sample  page 
from  the  booklet  appears  as  Figure  8.  The  booklet  was  similar  to  the 
rating  scales  successfully  used  by  Koonce  (1974)  to  measure  performance 
on  instrument  flight  manuevers. 

A  total  of  two  hundred  students  received  the  pretests,  flight  training, 
and  flight  tests.  Previous  flight  experience  of  subjects  varied  from  none 
to  private  certificates.  However,  no  cadets  had  received  undergraduate 
pilot  training,  and  average  flight  experience  among  VPI  students  versus 
cadets  was  approximately  equal. 

Because  approximately  600  hours  of  experimental  time  were  required 
over  a  short  period  of  time,  various  people  served  as  experimenters. 

At  VPI  pretesting  was  administered  by  Richard  E.  Becker,  Ricky  E.  Savage, 
David  0.  Cote,  and  Beverly  H.  Williges;  all  flight  training,  testing, 
and  rating  was  performed  by  Richard  E.  Becker. 

At  the  Air  Force  Academy,  pretesting,  training,  testing,  and  rating 
was  conducted  by  Lt.  Col.  Jefferson  M.  Koonce,  Richard  E.  Becker,  Lt.  Col. 
Gene  A.  Berry,  and  Charles  R.  Beaver.  Interrater  reliability  for  the  flight 
tasks  at  the  Air  Force  Academy  was  .893. 

All  statistical  analyses  were  conducted  at  VPI  using  the  University’s 
MVS  batch  services  on  twin  IBM  System/ 370  Model.  The  statistical  package 
used  was  SAS  76  by  Barr,  Goodnight,  Sail,  and  Helwig.  Richard  E.  Becker 
and  Scott  R.  Stacey  conducted  the  analyses  under  the  direction  of 
Lt.  Col.  Koonce  and  Dr.  Williges. 

Two-factor  analyses  of  variance  (ANOVA)  were  conducted  on  all  pretest 
data  to  determine  the  effects  of  sex  and  institution.  In  the  original 
pretest  battery  there  were  no  reliable  differences  due  to  institution. 


2. 


Descend  from  2000  fc.  to  1000  ft.  Heading  *  090  deg. 

V/V  ■  500  fpm..  Airspeed  «  160  mph.  Level  off  at  1000  ft. 
Once  descent  is  begun: 


Heading  h  I — 1 — 1 —  1 I I i  I  i  I  i  ci  deq 


20  to  0  10  20 


v/v 


lA"  rjx;  1  ^  1  I  I  I  .1  <♦!  fpm 

400  200  0  200  400 


Airspeed  <-) 


J _ I _ I _ 1  i  I 


•20  .10  0  10  20 


^ l  I  ciraph 


Rank 


<->  I _ I _ I - 1 _ I _ I _ I _ I _ I _ I _ l  (*l  deq 


'  20  10  0  10  20 


Level-off 

Altitude 


<->  I - 1 _ L-  1  ■ 


200  100  0  100  200 
(when  V/V  goes  to  or  through  zero) 


J  ci  ft 


TEST  3  DESCEND  1000  Ft. 


Figure  8.  Sample  page  from  rating  scale  booklet 
used  to  score  performance  on  the  ATC 
manuevers . 
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However,  females  scored  better  than  males  on  the  Identical  Pictures  Test 
(£■  .01),  and  males  scored  better  than  females  on  the  Cube  Comparison  Test 
(£>.0001),  Embedded  Figures  Test  (£>.0001),  and  pursuit  rotor  tracking 
(£>.0001).  On  all  PTD  tasks  males  performed  better  than  females  (£<.0001), 
and  Air  Force  cadets  performed  better  than  VPI  undergraduates  (£<.02). 

The  ANOVA  on  ATC  flight  test  ratings  revealed  reliable  main  effects 
of  sex,  £(1,196) >45. 55,  £<.0001,  and  institution,  £(1,196)>29.29,  £<.0009, 
indicating  that  males  performed  better  than  females  and  VPI  undergraduates 
performed  better  than  Air  Force  cadets.  The  reliable  interaction  of  sex 
and  institution,  F(l,196)>31.44,  £<.0006,  indicates  that  Air  Force  Academy 
females  performed  significantly  poorer  than  all  other  students  tested 
(see  Figure  9).  The  poor  showing,  on  the  average,  for  the  female  cadets 
seems  to  be  the  result  of  unusually  poor  performance  by  a  few  students.  In 
fact,  the  variability  among  female  cadets  was  twice  that  of  the  other 
student  groups. 

The  SAS  stepwise  linear  regression  procedure  was  used  to  determine 
prediction  equations  for  ATC  booklet  scores.  Table  3  summarizes  the  sample 
size,  multiple  R,  and  significant  predictors  for  various  samples.  Overall 
the  predictive  power  of  the  equations  was  disappointing.  However,  several 
trends  did  emerge.  First,  the  best  predictor  in  the  overall  equation  was 
institution  suggesting  that  separate  equations  for  each  school  are  desirable. 
When  separate  equations  were  developed  for  each  institution,  no  common 
predictors  occurred.  The  best  predictor  of  VPI  undergraduate  performance 
was  an  information  processing  test  (Map  Memory),  whereas  psychomotor  tests 
predicted  the  performance  of  the  cadets.  In  addition,  sex  was  the  best 
predictor  for  the  cadets'  performance,  indicating  that  separate  equations  for 
male  and  female  cadets  are  needed.  Indeed,  no  common  predictors  occurred 
when  separate  equations  were  developed  for  male  cadets  versus  female  cadets. 
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TABLE  3 

Sample  Size,  Multiple  R,  and  Significant  Predictors  of  Total  Booklet 
Score  for  the  Eight  ATC  Manuevers  for  Various  Samples 


Sample 

N 

R 

Significant  Predictors 
(£<•05) 

Combined 

200 

.44 

INST 

PTD2V  CC 

Males 

100 

.32 

PTD2Z 

Females 

100 

.49 

INST 

CC  PR 

Air  Force  Academy 

100 

.54 

SX 

PR 

PTD2V 

Males 

50 

.52 

PTD2Z 

IP 

Females 

50 

.36 

PR 

VPI 

100 

.36 

MM 

Males 

50 

.51 

MM 

PTD1 

Females 

50 

.33 

MM 

INST  *  Institution 

SX  =  Sex  of  Student 

CC  *  Cube  Comparison  Test 

IP  *  Identical  Pictures  Test 

MM  ■  Map  Memory  Test 

PR  »  Pursuit  Rotor 

PTD1  *  PTD  Test  1  -  Vector  error 

PTD2V  ■  PTD  Test  2  -  Vector  error.  Two-dimensional  tracking 
PTD2Z  ■  PTD  Test  2  -  Error,  One-dimensional  tracking 
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The  limited  predictive  ability  of  the  equations  may  be  related  to 
various  factors,  including  the  short  duration  of  training  and  the 
simplicity  of  the  flight  tasks.  In  addition,  our  previous  success 
with  the  regression  approach  involved  an  entirely  different  dependent 
variable  based  on  training  time  not  transfer  task  performance. 

Two  findings  from  this  research  are  Important  for  pilot  selection. 

First,  these  data  support  the  contentions  of  McGrevy  and  Valentine  (1974) 
that  only  the  second  test  on  the  PTD  is  consistently  a  reliable  predictor 
of  pilot  performance.  Second,  the  failure  to  reveal  any  conmon  predictors 
for  the  pilot  performance  of  male  and  female  cadets  suggests  that  some  caution 
is  appropriate  when  using  prediction  equations  developed  with  Air  Force 
males  to  predict  the  flight  performance  of  female  cadets. 


PUBLICATIONS 


During  the  contract  year  two  theses  were  completed,  four  papers  were 
presented,  and  two  papers  were  published.  These  papers  summarize  the 
research  related  to  augmented  feedback  in  adaptive  training,  the  regression 
approach  to  individual  differences,  and  the  prediction  of  pilot  performance. 
Citations  for  these  papers  and  presentations  are  given  below.  The  Appendix 
includes  a  copy  of  each  paper  or  an  abstract. 

Theses  Completed 

Savage,  R.  E.  A  multiple-regression  information-processing  approach  for 

assigning  individuals  to  a  training  strategy.  Masters  thesis.  Virginia 
Polytechnic  Institute  and  State  University,  February,  1979. 

Cote,  D.  0.  The  combined  effects  of  various  types  of  augmented  feedback 
and  two  training  procedures  on  motor  skill  learning.  Masters  thesis. 
Virginia  Polytechnic  Institute  and  State  University,  March,  1979. 

Papers  Presented 

Williges,  B.  H.  Computer  augmented  motor  skills  training.  Paper  presented 
at  the  International  Conference  on  Cybernetics  and  Society,  Tokyo- 
Kyoto,  Japan,  November,  1978. 

Williges,  R.  C.  and  Williges,  B.  H.  Automated  motor  skills  training  optimized 
for  individual  differences.  Paper  presented  at  the  Review  of  Air 
Force  Sponsored  Basic  Research,  Flight  and  Technical  Training,  United 
States  Air  Force  Academy,  March,  1979. 

Williges,  B.  H. ,  Williges,  R.  C.,  and  Savage,  R.  E.  Predicting  optimal 

training  group  assignment.  Paper  presented  at  the  23rd  annual  meeting 
of  the  Human  Factors  Society,  Boston,  Massachusetts,  October,  1979. 


Becker,  R.  J.,  Williges,  B.  H. ,  Villiges,  R.  C.,  and  Roonce,  J.  M. 

Prediction  of  performance  in  motor  skills  training.  Paper  presented 
at  the  23rd  annual  meeting  of  the  Human  Factors  Society,  Boston, 
Massachusetts,  October,  1979. 

Papers  Published 

Williges,  B.  H.  Computer  augmented  motor  skills  training.  Proceedings  of 
the  International  Conference  on  Cybernetics  and  Society,  Tokyo-Kyoto, 
Japan,  November  1978,  957-960. 

Williges,  B.  H.,  Williges,  R.  C, ,  and  Savage,  R.  E.  Predicting  optimal 

training  group  assignment.  Proceedings  of  the  23rd  annual  meeting  of 


the  Human  Factors  Society,  Boston,  Massachusetts,  October  1979,  295-299. 
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The  research  effort  was  directed  by  Dr.  Robert  C.  Wllliges  who  Is  a 
professor  of  industrial  engineering  and  operations  research  and  of  psychology 
at  Virginia  Polytechnic  and  State  University.  Dr.  Wllliges  was  assisted 
by  two  research  associates,  John  E.  Evans,  III  and  Beverly  H.  Wllliges, 
and  by  four  graduate  research  assistants,  Ricky  E.  Savage,  David  0.  Cote, 
Richard  J.  Becker,  and  Scott  Stacey.  Mr.  Evans  is  responsible  for  all 
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simulation  and  systems  programming  for  the  project.  Ms.  Wllliges  managed 
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and  Ms.  Wllliges  follow. 
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Professor  of  Industrial  Engineering 
Professor  of  Psychology 


EDUCATION 


A.B.,  Psychology,  1964  Wittenberg  University 

M.A.,  Psychology,  1966  The  Ohio  State  University 

Ph.D.,  Engineering  Psychology,  1968  The  Ohio  State  University 

EXPERIENCE 

Dr.  Williges  joined  the  staff  of  the  Department  of  Industrial  Engineering  and 
Operations  Research  in  1976.  Before  becoming  a  member  of  the  faculty  at  Virginia 
Polytechnic  Institute  and  State  University,  he  spent  eight  years  from  1968  to 
1976  as  a  member  of  the  Department  of  Psychology  at  the  University  of  Illinois 
at  Urbana-Champaign .  In  addition  to  his  teaching  appointment,  he  was  associated 
both  with  the  Highway  Traffic  Safety  Center  (1968-70)  where  he  conducted  research 
on  human  factors  applications  to  highway  systems  and  the  Aviation  Research 
Laboratory  (1970-76)  where  he  conducted  research  dealing  with  pilot  training 
and  enhancement  of  aircraft  controls  and  displays.  While  at  The  Ohio  State 
University  he  was  employed  at  the  Human  Performance  Center  (1964-68)  and 
conducted  simulation  research  on  air  traffic  control  systems  and  human  monitoring 
of  complex,  computer-generated  displays. 

Besides  his  extensive  experience  in  conducting  and  managing  human  factors  research, 
he  has  taught  both  graduate  and  undergraduate  courses  in  statistics,  research 
methodology,  industrial  psychology,  human  performance,  engineering  psychology, 
and  human  factors  in  systems  design.  His  publications  in  human  factors  include 
topics  dealing  with  team  training,  decision  making,  simple  and  complex  visual 
monitoring  performance,  inspection  behavior,  and  human  performance  in  complex 
system  operation  including  investigation  of  rate-field,  frequency-separated, 
predictor,  computer-generated,  and  time-compressed  displays,  interpretability 
of  TV-displayed  cartographic  information,  applications  of  response  surface 
methodology,  motion  cues  in  simulation,  transfer  of  training,  and  adaptive 
training  procedures. 


AFFILIATIONS  AND  AWARDS 

Human  Factors  Society  (Fellow,  1975):  editor,  Human  Factors  (1976),  associate 
editor,  Human  Factors  (1973-75),  reviewing  editor,  Human  Factors  (1971-73), 
publications  board  (1974-75),  and  president,  Sangamon  Valley  Chapter  (1971-72); 
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and  Operations  Research 


EDUCATION 


A.B.,  Psychology,  1965  (cum  laude) 
M.A. ,  Psychology,  1968 


Wittenberg  University 
The  Ohio  State  University 


EXPERIENCE 

Ms.  Williges  joined  the  staff  of  the  Human  Factors  Laboratory  as  a  research 
associate  in  1976.  Her  primary  responsibilities  include  the  development 
and  supervision  of  research  on  automated  training  systems,  the  design  of 
experiments,  statistical  analysis,  and  report  preparation.  From  1971-1976 
she  was  a  research  associate  at  the  Aviation  Research  Laboratory,  University 
of  Illinois,  where  she  was  involved  in  basic  and  applied  research  to  develop 
adaptive  systems  for  pilot  training.  Previously,  she  worked  for  Battelle 
Memorial  Institute,  Columbus  Laboratories,  where  she  was  responsible  for  the 
development  of  self-instructional  training  programs  to  be  used  in  industry, 
government,  and  education.  She  was  also  involved  in  the  collection,  evaluation, 
and  interpretation  of  large-scale  survey  data.  While  at  Battelle  she  attended 
a  short  course  at  the  University  of  Michigan  for  writers  of  programmed 
instruction.  Her  primary  research  interests  are  in  the  areas  of  motor  learning, 
computed-augmented  instruction,  and  simulation.  Her  publications  include 
self-instructional  training  programs  and  papers  dealing  with  research  on 
simulation,  learner-centered  instruction,  computer  adaptive  training,  and 
individual  differences  in  motor  learning. 

AFFILIATIONS  AND  AWARDS 

Human  Factors  Society:  Executive  Council,  member  (1976-1982);  Training 
Technical  Group,  steering  committee  (1975-1978),  chairperson  (1978-1979); 

Human  Factors:  Associate  editor  (1975);  consulting  editor  (1976-1980); 

Aviation  Research  Monographs:  Associate  editor  (1971-1973);  Alpha  Lambda 
Delta;  Psi  Chi;  Delta  Phi  Alpha;  Mortar  Board;  Charles  Platt  Award  in  Psychology. 

PUBLICATIONS 


Journal  Articles 

Slocum,  G.  K. ,  Williges,  B.  H.,  and  Roscoe,  S.  N.  Meaningful  shape  coding  for 
aircraft  switch  snobs.  Aviation  Research  Monograpsh.  1971,  JL,  (3), 

27-40. 

Roscoe,  S.  N.  and  Williges,  B.  H.  (Eds.)  Three  lessons  in  aviation  research. 
Aviation  Research  Monographs,  1971,  .1,  (3). 

Flexman,  R.  E.,  Roscoe,  S.  N.,  Williams,  A.  C.,  Jr.,  and  Williges,  B.  H. 
Studies  in  pilot  training:  The  anatomy  of  transfer.  Aviation  Research 
Monographs.  1972,  2.,  (1). 


tfilliges,  B.  H. ,  Roscoe,  S.  N. ,  and  Williges,  R.  C.  Synthetic  flight  training 
revisited.  Human  Factors.  1973,  .15,  543-560. 

Gopher,  D.,  Williges,  B.  H.,  Williges,  R.  C.,  and  Damos,  D.  L.  Varying  the 
type  and  number  of  adaptive  variables  in  continuous  tracking.  Journal 
of  Motor  Behavior.  1975,  2»  159-170. 

Williges,  B.  H.  and  Williges,  R.  C.  Learner-centered  versus  automatic 

adaptive  skill  training.  Journal  of  Motor  Behavior,  1977,  9.,  325-331. 

Williges,  R.  C.  and  Williges,  B.  H.  Critical  variables  in  adaptive  motor  skills 
training.  Human  Factors,  1978,  20,  201-214. 

Proceedings  Papers  and  Presented  Papers 

Hummel,  R.  L. ,  Williges,  B.  H. ,  and  Roscoe,  S.  N.  A  computer-generated  display 
to  isolate  essential  visual  cues  in  landing.  In  W.  B.  Knowles,  H.  S. 
Sanders,  and  F.  A.  Mucker  (Eds.)  Proceedings  of  the  sixteenth  annual 
meeting  of  the  Human  Factors  Society.  Santa  Monica,  Calif.:  Human 
Factors  Society,  October,  1972,  98-101. 

Gopher,  D. ,  Williges,  B.  H. ,  Williges,  R.  C.,  and  Damos,  D,  L.  Manipulating 
Che  number  and  type  of  adaptive  variables  in  training.  In  E.  L.  Saenger 
and  M.  Kirkpatrick  II  (Eds.)  Proceedings  of  the  eighteenth  annual  meeting 
of  the  Human  Factors  Society.  Santa  Monica,  Calif.:  Human  Factors 
Society,  October,  1974,  534-541. 


Williges,  B.  H.  and  Williges,  R.  C.  Manual  versus  automatic  adaptive  skill 
training.  Proceedings  of  the  fifth  annual  Symposium  on  Psychology  in 
the  Air  Force,  April,  1976,  31-35. 

Williges,  B.  H.  and  Williges,  R.  C.  Critical  variables  in  adaptive  training. 
Paper  presented  at  the  sixth  congress  of  the  International  Ergonomics 
Association,  College  Park,  Maryland,  July,  1976. 

Williges,  B.  H.,  Williges,  R.  C.,  and  Savage,  R.  E.  Models  for  automated  motor 
skills  training.  In  A.  S.  Neal  and  R.  F.  Palasek  (Eds.)  Proceedings  of 
the  twenty-first  annual  meeting  of  the  Human  Factors  Society.  Santa 
Monica,  California:  Human  Factors  Society,  October,  1977,  18-22. 

Williges,  B.  H.  S.  I.  (metric)  system  of  units:  why/how  to  institute? 

Panel  discussion  presented  at  the  21st  annual  meeting  of  the  Human 
Factors  Society,  San  Francisco,  California,  October,  1977. 

Williges,  B.  H.,  Williges,  R.  C. ,  and  Savage,  R.  E.  Matching  initial  performance 
and  the  measurement  of  sex  differences.  Proceedings  of  the  sixth  symposium 
on  Psychology  in  the  DoD.  Colorado  Springs,  Colorado:  U.  S.  Air  Force 
Academy,  April,  1978. 

Savage,  R.  E.,  Williges,  R.  C.,  and  Williges,  B.  H.  Individual  differences  in 
motor  skill  training.  Proceedings  of  the  sixth  symposium  on  Psychology 
in  the  DoD.  Colorado  Springs,  Colorado:  0.  S.  Air  Force  Academy,  April, 
1978. 
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Williges,  R.  C.  and  Williges,  B.  H.  Automated  motor  skills  training  optimized 
for  individual  differences.  Paper  presented  at  the  Review  of  Air  Force 
Sponsored  Basic  Research:  Flight  and  Technical  Training,  U.S.  Air  Force 
Academy,  April,  1978. 

Cote,  0.  0.,  Williges,  B.  H.,  and  Williges,  R.  C.  Augmented  feedback  in 

adaptive  motor  skill  training.  In  E.  J.  Baise  and  J.  M.  Miller  (Eds.) 
Proceedings  of  the  22nd  annual  meeting  of  the  Hum*"  Factors  Society. 

Santa  Monica,  Calif.:  The  Human  Factors  Society,  October,  1978, 

105-109. 

Savage,  R.  E. ,  Williges,  R.  C.,  and  Williges,  B.  H.  Cross-validation  of 
regression  equations  to  predict  performance  in  a  pursuit  tracking 
task.  In  E.  J.  Baise  and  J.  M.  Miller  (Eds.)  Proceedings  of  the 
22nd  annual  meeting  of  the  Human  Factors  Society.  Santa  Monica,  Calif.: 
The  Human  Factors  Society,  October,  1978,  369-372. 

Williges,  B.  H.  Computer- augmented  motor  skills  training.  Proceedings  of  the 
International  Conference  on  Cybernetics  and  Society.  Tokyo-Kyoto,  Japan, 
November,  1978,  957-960. 

Williges,  R.  C.  and  Williges,  B.  H.  Automated  motor  skills  training  optimized 
for  individual  differences.  Paper  presented  at  the  Review  of  Air  Force 
Sponsored  Basic  Research:  Flight  and  Technical  Training,  U.S.  Air  Force 
Academy,  April,  1979. 

Williges,  B.  H. ,  Williges,  R.  C.,  and  Savage,  R.  E.  Predicting  optimal 

training  group  assignment.  In  C.  K.  Bensel  (Ed.)  Proceedings  of  the 
23rd  annual  meeting  of  the  Human  Factors  Society.  Santa  Monica,  Calif.: 
The  Human  Factors  Society,  October,  1979,  295-299. 

Technical  Reports 

Williges,  B.  H.  Basic  stocking:  Self-instruction  manual.  Cincinnati, 

Ohio:  The  Kroger  Company,  1967. 

Coffey,  J.  L.,  Cress,  R.  J.,  Williges,  B.  H.,  and  Swedberg,  S.  G.  Evaluating 
the  efficiency  and  effectiveness  of  self-instructional  methods  for 
selected  areas  of  vocational  education.  Columbus,  Ohio:  Battelle 
Memorial  Institute,  Final  Report,  U.S.  Office  of  Education  Grant  No. 
0EG-36-000539-1215 ,  February,  1968. 

Williges,  R.  C.  and  Williges,  B.  H.  (Eds.)  Selected  papers  on  response 

surface  methodology.  Savoy,  Illinois:  Aviation  Research  Laboratory, 
Institute  of  Aviation,  University  of  Illinois  at  Urbana-Champaign, 
Technical  Report  ARL-72-20/AF0SR-72-9,  August,  1972. 

Williges,  B.  H.  and  Williges,  R.  C.  Automated  motor  skills  training 

optimized  for  individual  differences:  Annual  report  October  1976  - 
September  1977.  Blacksburg,  Virginia:  Virginia  Polytechnic  Institute 
and  State  University,  HFL-77-1/AFOSR-77-1,  November,  1977. 

Williges,  B.  H.  and  Williges,  R.  C.  Automated  motor  skills  training 

optimized  for  individual  differences:  Annual  Report,  October  1977  - 
September  1978.  Blacksburg,  Virginia:  Virginia  Polytechnic  Institute 
and  State  University,  HFL-78-5/AFOSR-78-1,  November,  1978. 
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JOHN  E.  EVANS,  III  Research  Associate 

Human  Factors  Laboratory 
Department  of  Industrial  Engineering 
and  Operations  Research 

EDUCATION 

Virginia  Polytechnic  Institute 
and  State  University 
Virginia  Polytechnic  Institute 
and  State  University 


B.S.,  Electrical  Engineering, 
1974  (with  distinction) 

M.S.,  Computer  Science  and 
Applications,  1975 


EXPERIENCE 


Mr.  Evans  joined  the  staff  of  the  Human  Factors  Laboratory  as  a  research  associate 
in  1976.  He  has  since  been  involved  in  systems  software  modifications,  corrections, 
and  enhancements,  and  applications  software  development  including  real-time 
simulation,  dynamic  computer  graphics,  data  reduction,  real-time  data  acquisition, 
computer  transaction  systems  design  and  implementation,  program  debugging  aids 
and  instrumentation,  and  on-line  experiment  control  using  interconnected  DEC 
POP  11/10  and  11/55  mini-computers.  In  addition,  he  has  been  involved  in  systems 
and  applications  software  development  in  the  area  of  magnetic  tape  file  pro¬ 
cessing,  program  debugging  aids,  and  program  library  maintenance  on  the  IBM 
370/158  university  computers.  His  laboratory  support  responsibilities  encompass 
many  aspects  of  digital  computation  including  software  maintenance,  software 
design,  development,  and  documentation,  computer  facilities  planning,  hardware/ 
software  evaluation  and  acquisition,  hardware  failure  diagnosis,  programner 
consultation,  interface  design  for  custom  laboratory  peripherals,  and  technical 
wri ting. 

Specific  areas  in  which  major  software  development  has  been  concentrated  are  eye 
cracking  data  analysis,  adaptive  motor  skills  training,  operator  performance 
measurement  in  computer  transaction  systems,  DEC/IBM  magnetic  tape  data  transfer, 
device  specific  graphics,  software  debugging  aids,  special  purpose  device  handlers, 
and  automated  data  acquisition.  In  his  current  position,  Mr.  Evans  has  parti¬ 
cipated  in  several  funded  research  programs  in  the  capacity  of  a  software  support 
engineer.  Also,  he  is  responsible  for  the  supervision  of  work  assignments  and 
performance  review  of  one  systems  analyst. 

From  1975-1976  Mr.  Evans  was  a  systems  analyst  in  the  Technical  Computer  Systems 
Development  Group  at  Shell  Information  Center,  Shell  Oil  Company  in  Houston, 

Texas,  where  he  was  Involved  in  systems  software  development,  program  instrumentation 
and  systems  measurement  and  performance  evaluation  for  UNIVAC  1100  series  multi¬ 
processor  computers  used  in  geophysical  and  scientific  applications  programming. 
Through  the  analysis  of  the  effects  of  the  introduction  of  artificial  Job  mixes 
to  the  existing  programing  environments,  he  conducted  and  reported  several  system 
impact  and  utilization  studies  of  projected  UNIVAC  system  loading.  In  addition, 
he  was  involved  in  utility  and  user  aid  software  development  for  the  VM/CMS 
system  operating  on  an  IBM  370/158. 

Previously,  Mr.  Evans  worked  as  a  graduate  research  assistant  in  the  Computer 
Engineering  Laboratory,  Department  of  Electrical  Engineering,  Virginia  Polytechnic 
Institute  and  State  University,  where  he  developed  a  software  system  (loaders, 
utilities,  device  handlers,  cross-assembler,  cross-compiler,  object  time  support 


library,  instruction  tracer,  and  interactive  debugger)  for  the  GE/PAC  30-2E 
minicomputer  (Interdata  5  equivalent) .  He  also  developed  an  artificially 
intelligent  3-D  4x4x4  tic-tac-toe  re-entrant  game  player  (95%  win  rate) 
for  the  GE/PAC  30-2E.  In  addition,  he  developed  an  extensive  Fortran  callable 
subprogram  library  providing  many  processing  extensions  to  ANSI  Fortran  IV  on 
the  IBM  360  operating  under  OS.  Additional  programming  included  several 
utilities  and  Fortran  callable  subprogram  packages  under  OS  and  a  large  amount 
of  user-aid  software  under  TSO  in  support  of  the  laboratory. 

During  his  graduate  study,  Mr.  Evans  served  on  the  Computer  Resources  Committee 
in  the  Department  of  Computer  Science.  As  an  undergraduate,  he  was  hired  as  a 
student  programmer  on  the  Naval  Weapons  Laboratory  Logic  Simulator  Program  CDC 
to  IBM  conversion  project.  He  also  participated  in  the  Cooperative  Education 
program  as  a  technical  assistant  electrical  engineer  with  Virginia  Electric 
and  Power  Company  where  he  was  involved  in  the  processing  of  electric  service 
requests,  power  distribution  planning,  protection,  and  design,  power  trans¬ 
mission  planning  and  protection,  substation  design,  and  automation  of  calculations 
for  power  system  protection.  He  holds  an  Engineer  in  Training  certificate  in 
the  state  of  Virginia  and  is  currently  working  on  a  Ph.D.  in  Computer  Science 
and  Applications.  His  primary  research  interests  are  artificial  intelligence, 
compiling  techniques,  operating  systems,  and  software  transportability. 

With  a  programming  career  total  in  excess  of  85,000  lines,  Mr.  Evans  has  extensive 
experience  with  IBM  360/370  under  OS,  MVS,  and  VM/CMS,  UNIVAC  1108/1110  under 
EXEC  8,  and  DEC  PDP  11  under  RT-11.  Programming  languages  in  which  he  is  pro¬ 
ficient  are  Fortran  IV/V  (most  compiler  dialects) ,  PL/1,  and  Assembly  (IBM 
360/370,  UNIVAC  1108/1110,  DEC  PDP  11,  Interdata  5).  In  addition,  he  is,  to 
at  least  a  reading  understanding,  familiar  with  a  large  number  of  other  commonly 
encountered  programming  languages,  job  control  languages,  operator  command 
languages,  interactive  operating  system  user  languages,  etc.  Recent  job  independen 
projects  have  included  a  more  powerful  and  intelligent  3-D  4x4x4  tic-tac-toe 
game  player  on  IBM  VM/CMS,  a  support  package  and  conversion  rules  for  translating 
most  CDC  Fortran  programs  to  the  IBM  360/370  computer  in  a  line-by-line  context 
independent  manner,  and  an  interactive  line  extraction  program  for  use  with  grey 
level  imagery  on  the  HP  2100A  minicomputer. 

AFFILIATIONS  AND  AWARDS 

Eta  Kappa  Nu;  Tau  Beta  Pi;  Kappa  Theta  Epsilon  (Cooperative  Education  Honor 
Society);  Upsilon  Pi  Epsilon;  The  Lepidopterists'  Society. 

PUBLICATIONS 

Journal  Articles: 

Evans,  J.  E.,  Ill  and  Gutmann,  J.  C.  Minicomputer  processing  of  dual  Purkinje 
Image  eye  tracker  data.  Behavior  Research  Methods  and  Instrumentation, 
October,  1978. 

Presented  Papers: 


Gutmann,  J.  C.,  Snyder,  H.  L.,  Farley,  W.  W,  and  Evans,  J.  E.,  III.  The 
effect  of  image  quality  on  search  for  s.atic  and  dynamic  targets: 
MTFA-performance  correlations.  Proceedings  of  the  23rd  annual  meeting 
of  the  Human  Factors  Society,  Boston,  Massachusetts,  October,  1979. 
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Gutmann,  J.  C.,  Evans,  J.  E.,  Ill,  Farley,  W.  W. ,  and  Snyder,  H.  L.  Eye 
movement  measures  as  Indices  of  visual  display  image  quality.  Paper 
presented  at  the  American  Psychological  Association  annual  convention, 
Toronto,  Canada,  September,  1978. 

Gutmann,  J.  C. ,  Evans,  J.  E.,  Ill,  Farley,  W.  W.,  and  Snyder,  H.  L.  Eye 
movement  measurements  as  indices  of  visual  display  image  quality. 

Paper  presented  at  the  21st  annual  meeting  of  the  Human  Factors  Society, 
San  Francisco,  California,  October,  1977. 

Technical  Reports; 

Gutmann,  J.  C.,  Snyder,  H.  L.,  Farley,  W.  W. ,  and  Evans,  J.  E.,  III.  An 
experimental  determination  of  the  effect  of  image  quality  on  eye 
movements  and  search  for  static  and  dynamic  targets.  Dayton,  OH: 

Aerospace  Medical  Research  Laboratory,  Wright-Patterson  AFB,  Technical 
Report  AMRL-TR- 79-51,  1979. 

Evans,  J.  E.,  Ill,  and  Mason,  L.  H.  Generic  data  transaction  system:  version 
2  system  design.  Blacksburg,  VA:  Virginia  Polytechnic  Institute  and 
State  University,  Human  Factors  Laboratory,  Department  of  Industrial 
Engineering  and  Operations  Research,  Technical  Report  HFL-79-2/NPRDC-79-1, 
March,  1979. 

Evans,  J.  E.,  Ill  and  Mason  L.  H.  Generic  data  transaction  system:  version  2 
system  source.  Blacksburg,  VA:  Virginia  Polytechnic  Institute  and 
State  University,  Human  Factors  Laboratory,  Department  of  Industrial 
Engineering  and  Operations  Research,  Technical  Report  HFL-79-3/NPRDC-79-2, 
March,  1979. 

Evans,  J.  E.,  Ill  and  Mason,  L.  H.  Generic  data  transaction  system:  system 
design.  Blacksburg,  VA:  Virginia  Polytechnic  Institute  and  State 
University,  Human  Factors  Laboratory,  Department  of  Industrial  Engineering 
and  Operations  Research,  Technical  Report  HFL-78-2/NPRDC-78-2 ,  July,  1978. 

Evans,  J.  E.,  III.  Introduction  to  the  NSWC  logic  simulator  program  CDC/IBM 
Fortran  conversion.  Blacksburg,  VA:  Virginia  Polytechnic  Institute 
and  State  University,  Computer  Engineering  Laboratory,  Department  of 
Electrical  Engineering,  1977. 

Evans,  J.  E.,  III.  CDC/IBM  Fortran  conversion  for  the  NSWC  logic  simulation 
program.  Blacksburg,  VA:  Virginia  Polytechnic  Institute  and  State 
University,  Computer  Engineering  Laboratory,  Department  of  Electrical 
Engineering,  1977. 

Evans,  J.  E.,  III.  A  software  system  study  of  the  GE/PAC  30-2E  minicomputer. 
Blacksburg,  VA:  Virginia  Polytechnic  Institute  and  State  University, 
Computer  Engineering  Laboratory,  Department  of  Electrical  Engineering, 
1975. 
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INTRODUCTION 


When  developing  a  training  program,  a  trainer  needs  to  maximize 
training  efficiency  because  of  such  constraints  as  budget,  training 
time,  and  need  for  highly  skilled  trainees.  However,  when  developing 
a  training  program  for  motor  skills,  he  has  no  definitive  theory  or 
model  to  serve  as  a  guideline.  Therefore,  the  trainer  must  choose 
between  two  competing  approaches:  (1)  selecting  an  optimal  training 
model  designed  for  the  average  trainee,  or  (2)  selecting  an  optimal 
training  model  designed  for  the  individual. 

If  the  choice  is  the  former,  the  trainer  will  encounter  a 
fundamental  difficulty:  this  optimal  training  model  will  not  meet  the 
needs  of  the  individual.  Some  students  will  perform  well  using  this 
particular  training  model,  but  others  will  have  difficulty  even  though 
they  may  be  good  students.  The  same  problem  can  result  even  when  the 
training  model  allows  some  flexibility  or  individual  adjustments  for 
various  skill  levels  of  the  student  (e.g.,  adaptive  training,  Kelley, 
1969)  since  any  adjustments  will  have  to  be  based  on  the  same  logic 
system  for  all  individuals.  Generally,  this  flexibility  is  not  broad 
enough  because  individual  factors  such  as  learning  style,  information- 
processing  capabilities,  etc.,  are  not  taken  into  consideration.  The 
basic  problem  with  this  approach  is  that  the  training  model  is  not 
necessarily  matched  to  the  individual  characteristics  of  the  student. 

The  alternative  approach  is  to  select  a  training  model  that  matches 
certain  characteristics  of  the  student  to  that  model.  Glaser  (1970) 
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stated  that  in  order  to  maximize  training  effectiveness  for  the 

individual,  the  goal  of  training  research  should  be  to  determine  inter- 

*  *  % 

actions  between  individual  measurements  and  training  strategies. 

Recent  work  on  cognitive  tasks  by  Pask  (1976)  supports  this  approach. 

Pask  found  that  when  the  student's  preferred  learning  style  and  teaching 
strategy  are  mismatched,  comprehension  and  retention  are  severely 
disrupted. 

The  research  to  be  reported  in  this  thesis  follows  the  suggestion 
of  Glaser  (1970)  in  trying  to  match  a  particular  training  strategy  to 
individual  characteristics.  The  initial  task  in  proceeding  with  such 
research  was  to  discover  which  individual  characteristics  relate 
significantly  to  a  particular  training  model  and  to  establish  a  method 
for  selecting  a  training  strategy  for  the  individual  based  on  these 
characteristics . 

Components  and  Models  of  Motor  Learning 

In  determining  individual  characteristics  which  may  relate  to  a 
particular  training  strategy,  the  trainer  has  two  strategies  to  consider: 
the  components  of  motor  learning  or  a  theory  (or  model)  of  motor  learning. 
The  components  of  motor  learning  could  consist  of  motor  abilities  or 
nonmotor  factors  which  relate  to  motor  skill  learning.  By  measuring 
these  components,  the  trainer  can  discover  relationships  between 
individual  characteristics  and  various  training  strategies. 

Fleishman  and  his  colleagues  (1954,  1958,  1964,  1972)  have  probably 
studied  the  components  of  motor  learning  more  than  any  other  researchers. 
The  emphasis  of  their  research  was  on  motor  abilities  using  correlational 
and  factor-analytic  approaches.  From  administering  some  200  psychomotor 


tests  to  thousands  of  subjects,  Fleishman  and  his  associates  can  account 

for  performance  on  the  tasks  in  terms  of  ^  small  number  of  motor 

.  • . 

abilities  (such  as  coordination,  reaction  time,  dexterity,  etc.)  and 
physical-proficiency  abilities  (such  as  static  and  dynamic  strength, 
stamina,  extent  flexibility,  etc.). 

In  the  area  of  nonmotor  factors  and  their  influence  upon  motor 
skills,  the  research  is  rather  limited.  However,  some  aspects  of 
personality  (e.g. ,  Ismail,  Kane,  and  Kirkendall,  1969,  extroversion- 
introversion  and  stability-neuroticism) ,  stress  (e.g.,  Carron,  1968), 
and  motivation  (e.g.,  Rushall  and  Pettinger,  1969)  have  been  related  to 
motor  skill  learning. 

A  second  procedure  for  determining  relationships  between  individual 
characteristics  and  a  particular  training  strategy  consists  of  measuring 
certain  processes  or  mechanisms  of  a  theory  (or  model)  of  motor  learning. 
Some  examples  consist  of  Adams's  (1968,  1971)  closed-loop  theory, 
Whiting's  (1969,  1972)  systems  model  of  skilled  performance,  or  the 
information-processing  models  of  Welford  (1968),  Singer  (1975),  and 
Marteniuk  (1976) . 

Several  of  these  components  of  motor  learning  (e.g.,  Fleishman, 

1972)  could  be  used  to  determine  relationships  with  various  training 
strategies.  Much  of  the  emphasis  of  information  processing  in  motor 
skills,  particularly  by  Welford  (1968)  and  Marteniuk  (1976),  is  that 
limitations  in  any  of  these  mechanisms  and  their  various  components  can 
limit  performance.  Although  they  rarely  discuss  individual  differences 
specifically,  the  implications  are  obvious.  If  any  of  these  perceptual 
or  cognitive  processes  involved  in  information  processing  can  be  measured 
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to  determine  differences  in  capacities,  then  individual  differences 

can  be  studied  in  how  they  relate  to  various  training  strategies. 

.  ». 

Methods,  Procedures ,  and  Decision  Rules 

Having  found  measurable  factors  which  relate  to  a  particular 
training  strategy,  the  trainer  needs  to  determine  a  method  or  procedure 
which  will  provide  a  decision  rule.  This  decision  rule  would  select  a 
training  strategy  based  on  the  measures  taken  from  the  individual.  The 
development  of  a  decision  rule  usually  involves  some  type  of  prediction 
or  heuristic  model.  This  methodological  problem  will  now  be  considered. 

Although  there  is  no  definite  solution  to  this  problem,  several 
alternative  procedures  and  methods  exist.  For  example,  Fleishman  (1972) 
uses  primarily  correlational  or  factor-analytic  techniques  to  determine 
significant  components  of  motor  learning;  however,  both  of  these 
procedures  offer  little  in  the  way  of  a  decision  rule.  The  trainer 
could  use  these  procedures  to  determine  relationships  with  a  particular 
training  strategy,  but  he  would  be  forced  to  develop  a  heuristic  method 
to  serve  as  a  decision  rule. 

The  technique  of  discriminant  analysis  offers  the  basis  for  a 
decision  rule.  This  statistical  procedure  will  find  not  only  significant 
factors  that  are  related  to  a  training  strategy,  but  also  classify  an 
individual  into  one  of  several  categories  or  training  strategies.  For 
purposes  of  designing  a  training  program  based  on  individual  differences, 
discriminant  analysis  has  definite  possibilities.  The  basic  problem 
with  discriminant  analysis  is  the  lack  of  performance  information  within 
each  training  strategy.  Discriminant  analysis  will  find  individual 


characteristics  that  relate  to  a  particular  training  strategy,  but  not 

to  performance  within  that  training  strategy.  Therefore,  the  lack  of 

• . 

performance  information  can  restrict  the  use  of  discriminant  analysis 
as  a  decision  rule. 

Cronbach  (1975)  and  Cronbach  and  Snow  (1977)  propose  a  procedure 
which  they  call  "Aptitude-Treatment-Interaction"  (ATI) .  These  researchers 
of  individual  differences  firmly  suggest  the  need  to  match  characteristics 
of  the  individual  with  the  instructional  strategy.  Cronbach  suggests 
the  use  of  regression  analysis  to  determine  ATI.  By  testing  the  signifi¬ 
cance  of  the  beta  weights  of  some  aptitude  on  two  (or  more)  treatments, 
the  existence  of  an  ATI  can  be  determined.  An  ATI  effect  would  indicate 
that  different  aptitude  scores  would  result  in  differential  performance 
in  the  two  treatments  (or  training  strategies).  Cronbach  further  suggests 
using  higher  order  interactions  (i.e.,  more  than  one  aptitude)  to 
determine  a  number  of  factors  that  significantly  relate  to  various 
instructional  strategies.  A  specific  decision  rule  is  not  suggested  by 
Cronbach;  however,  ATI  does  provide  an  effective  method  for  determining 
relationships  between  individual  characteristics  and  training  strategies. 
By  using  the  ATI  approach,  the  trainer  is  again  faced  with  developing  a 
heuristic  decision  rule. 

Conway  and  Norman  (1974)  have  recently  suggested  a  rather  comprehen¬ 
sive  and  complex  procedure  which  may  serve  as  a  decision  rule.  They 
propose  a  self-organizing  training  system  capable  of  identifying 
different  learning  styles.  This  system  is  based  on  a  performance 
profile  and  a  personal  history  of  the  individual.  Some  of  the  suggested 
parameters  for  determining  individual  differences  include  previous 
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experience,  personality  characteristics,  cognitive  components,  psycho- 
motor  ability,  and  the  individual's  social  and  educational  background. 
Vreuls,  Woolridge,  Conway,  Johnson,  Freshour,  and  Norman  (1977)  have 
applied  some  of  these  concepts  in  the  development  of  a  higher-order, 
partially  self-organizing  adaptive  flight  training  system.  However, 
this  project,  still  in  the  initial  stages,  needs  further  research  to 
determine  its  applicability. 

Recently,  another  method  was  outlined  by  Savage,  Williges,  and 
Williges  (1978)  in  selecting  a  training  strategy,  based  on  individual 
measures,  to  train  students  on  a  two-dimensional  pursuit  tracking  task. 
These  authors  propose  a  three-phase  procedure  leading  to  the  development 
of  a  decision  rule.  The  first  phase  consists  of  developing  multiple- 
regression  equations  to  predict  time  to  train  performance  in  two 
different  training  models.  The  second  phase  involves  the  double  cross- 
validation  of  these  regression  equations  with  a  second  sample  of 
subjects.  The  final  phase  is  the  actual  use  of  the  predicton  equations 
in  selecting  a  training  strategy  for  the  individual.  This  last  phase 
would  involve  matching,  mismatching,  and  randomly  assigning  subjects 
to  a  training  condition.  The  uniqueness  of  this  approach  is  that 
individuals  are  not  being  classified  into  one  category  or  another  (or 
one  training  strategy  as  opposed  to  another).  Rather,  an  individual's 
performance  is  assessed  by  obtaining  predicted  time  to  train  scores  in 
the  various  training  conditions  from  the  regression  equations.  By  using 
this  information,  a  trainer  can  assign  an  individual  to  a  training 
strategy  which  results  in  a  savings  of  training  time. 


Results  of  the  First  Phase 

Savage,  et  al.  (1978)  reported  the  results  of  the  first  phase. 

.  •- 

The  research  to  be  reported  in  this  thesis  is  an  extension  of  these 
results;  therefore,  the  first  phase  data  will  be  discussed  in  some 
detail. 

These  regression  equations  were  based  primarily  on  information- 

processing  variables  from  Marteniuk’s  (1976)  model.  In  general,  three 

highly  reliable  regression  equations  predicting  time  to  exit  (i.e., 

time  to  train)  were  found  for  each  training  condition.  Table  1  presents 

the  regression  equations  along  with  the  level  of  significance  (p-value) , 
2 

the  multiple  R  (percent  of  accounted  for  variance) ,  the  estimate  of 
2 

shrinkage,  Rs  (Kerlinger  and  Pedhazur,  1973) ,  and  the  number  of  subjects 
in  each  equation,  n.  The  predictor  variables  are  in  standard  score 
format . 

The  two  regression  equations  labeled  "overall"  used  all  of  the 
subjects  in  each  training  condition.  Seven  predictor  variables,  listed 
at  the  bottom  of  Table  1,  were  used.  As  can  be  seen,  the  subject's  sex 
was  an  important  variable  only  in  adaptive  training.  The  next  two 
equations  were  developed  using  only  the  male  or  female  subjects  within 
their  respective  training  condition.  With  the  exception  of  the  fixed 
training  female  subjects,  these  equations,  based  on  the  sex  of  the 
subject,  account  for  a  larger  proportion  of  variance  than  the  overall 
equations.  Furthermore,  different  sets  of  predictor  variables,  not 
just  different  weightings,  are  used  in  the  regression  equations  based 
on  the  sex  of  the  subject.  By  developing  separate  regression  equations 
for  each  sex.  Savage,  et  al.  (1978)  felt  that  more  variance  could  be 
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TABLE  1 

Regression  Equations  for  Training  Time-to-Exit  by  Training  Condition 

.  • . 

Adaptive/Overall  (n  =  31) 

TE  «  1446=13  +  410.08  EF  +  263.93  SE  -  253.19  MM 

R2  =  0.721 

R2  =  0.690 
s 

p  -  0.0001 

Adaptive/Male  (n  =  18) 

TE  =  1069.50  +  509.86  EF  -  325.06  MM 
2 

R  *  0.750 
2 

R  *  0.717 
s 

p  *  0.0001 

Adaptive/Female  (n  =  13) 

TE  *  1967.61  -  492.83  MM  +  391.02  IP  -  358.42  CC  +  196.43  PR 

R2  =  0.871 

R2  =  0.806 
s 

p  =  0.0001 

Fixed/Overall  (n  *  28) 

TE  =  1160.73  +  509.20  EF  +  317.78  IP 

R2  =*  0.639 
2 

R  =»  0.610 

S 


p  =  0.0001 


TABLE  1 — Continued 


Fixed/Male  ( n  =  16)  *  *- 

TE  =  867.84  +  412.35  IP  +  333.12  EF 
2 

R  »  0.803 
2 

R  =  0.758 
s 

p  *  0.0001 

Fixed/Female  (n  =  12) 

TE  =  1551.25  +  411.87  EF 
2 

R  =  0.431 

R2  =  0.374 
s 

p  =  0.0205 

TE  -  Training  Time-to-Exit,  seconds 

PR  =  Pursuit  Rotor  (continuous  motor  skill) ,  time  on  target 

EF  =  Embedded  Figures  Test  (field  independence),  time  to  locate  figure 

MM  =  Map  Memory  Test  (visual  memory) ,  number  correct  minus  number  incorrect 

IP  =  Identical  Picutres  Test  (perceptual  speed) ,  number  correct  minus 
number  incorrect 

CC  =  Cube  Comparison  Test  (spatial  orientation) ,  number  correct  minus 
number  incorrect 


MT  =  Maze  Tracing  Test  (spatial  scanning) ,  number  correct  minus  number 
incorrect 
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accounted  for  because  of  performance  differences  found  in  the  pursuit 

tracking  task.  The  results  in  Table  1  generally  support  this  notion. 

.  • . 

Williges,  Williges,  and  Savage  (1978)  report  data  collected  on 
regression  equations  which  attempt  to  predict  mean  root-mean-square 
(RMS)  error  during  a  seven-minute  transfer  task.  This  task  involved 
the  same  apparatus  used  by  Savage,  et  al.  (1978).  The  researchers  used 
the  same  format — male,  female,  overall — with  these  equations  as  they 
had  used  in  the  previous  study.  These  regression  equations  are 
presented  in  Table  2. 

In  general,  these  regression  equations  do  not  account  for  as  much 
variance  as  did  the  regression  equations  predicting  time  to  exit 
performance  in  the  Savage,  et  al.  (1978)  study.  The  equations  generated 
for  the  female  subjects  were  the  exceptions  in  that  these  equations  were 
generally  highly  reliable  and  accounted  for  a  large  proportion  of 
variance.  Williges,  et  al.  (1978)  stated  that  this  finding  may  indicate 
that  task-specific  factors  dominate  for  male  subjects,  whereas  general 
ability  factors  are  characteristic  for  female  subjects  in  the  transfer 
task.  Since  these  predictor  equations  use  information-processing 
variables,  it  may  account  for  the  high  predictability  for  female  subjects 
as  opposed  to  male  subjects.  Therefore,  it  appears  that  male  subjects 
may  be  at  a  different  stage  of  training  in  the  transfer  task  and  that 
task-specific  variables  are  evident,  whereas  female  subjects  are  not  at 
this  stage  of  training  in  that  general  abilities  are  still  characteristic. 

Williges,  et  al.  (1978)  attempted  to  predict  performance  in  the 
transfer  task  because  a  decision  rule  based  on  a  savings  of  training 


TABLE  2 


Regression  Equations  Predicting  Mean  RMS  Vector  Error  by  Training 

.  •- 

Condition 


Adaptive/Overall  (n  ■  31) 

RMS  -  0.1074  +  0.0162  EF  +  0.0127  PR  -  0.0127  MT  -  0.0109  MM  +  0.0090  SE 
2 

R  -  0.691 

R2  *  0.629 
s 

p  =  0.0001 

Adaptive/Male  (n  =  18) 

RMS  »  0.0985  +  0.0186  EF  +  0.0093  IP 

R2  =*  0.545 

R2  =  0.484 
s 

p  =  0.0027 

Adaptive/Female  (n  *  13) 

RMS  =  0.1197  -  0.0352  MM  -  0.0187  MT  +  0.0175  FR 

R 2  =  0.906 

R2  =•  0.875 
s 

p  =>  0.0009 

Fixed/Overall  (n  =  28) 

RMS  *  0.1090  -  0.0064  SE  +  0.0064  EE 
R2  -  0.452 
R2  =  0.408 

S 


p  =  0.0005 
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TABLE  2 — Continued 

Fixed/Male  (n  =  16)  •  *- 

No  reliable  equation  found 
Fixed/Female  (n  =  12) 

RMS  -  0.1189  -  0.0076  IP  -  0.0071  PR  +  0.0065  MT  -  0.0044  MM 
2 

R  -  0.807 
2 

R  =  0.697 
s 

p  *  0.0121 

RMS  =  Root-Mean-Square  vector  error 

PR  =  Pursuit  Rotor  (continues  motor  skill) ,  time  on  target 

EF  =  Embedded  Figures  Test  (field  independence) ,  time  to  find  figure 

MM  =  Map  Memory  Test  (visual  memory) ,  number  correct  minus  number 
incorrect 

IP  =  Identical  Pictures  Test  (perceptual  speed) ,  number  correct  minus 
number  incorrect 

CC  -  Cube  Comparison  Test  (spatial  orientation) ,  number  correct  minus 
number  incorrect 

MT  =  Maze  Tracing  Test  (spatial  scanning) ,  number  correct  minus  number 
incorrect 


SE  =  Sex 
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time  may  not  be  the  most  desirable  or  efficient  method.  How  well  an 

individual  performs  in  a  transfer  task  could  be  the  basis  for  a 

.  •  - 

decision  rule.  This  basis  for  a  decision  rule  is  often  of  more 
importance  to  a  trainer  than  a  savings  of  training  time. 

Summary 

The  Savage/  et  al.  (1976)  and  Williges,  et  al.  (1968)  studies 
resulted  in  three  principal  conclusions.  First,  information-processing 
variables,  used  as  predictors,  are  significant  in  predicting  pursuit 
tracking  performance.  Information-processing  variables  are  proposed  in 
several  human  performance  models,  but  they  have  not  received  thorough 
investigation  to  determine  their  importance  in  motor  skill  learning. 

These  two  studies  suggest  their  validity  as  topics  for  research.  Second, 
regression  equations,  as  opposed  to  other  methods,  were  successfully 
used  to  predict  pursuit  tracking  performance  and  to  determine  significant 
predictors  of  pursuit  tracking  performance.  Again,  research  has  been 
lacking  on  the  viable  use  of  regression  analysis  to  study  individual 
differences  in  motor  skill  learning;  these  studies  point  to  a  successful 
procedure  which  can  be  utilized.  Finally,  a  three-phase  methodological 
approach  for  determining  decision  rules  was  introduced.  Therefore,  the 
purpose  of  the  current  research  is  to  carry  out  the  second  and  third 
phases  of  this  approach.  Study  I  determines  the  validity  of  these 
regression  equations  presented  by  Savage,  et  al.  (1978)  and  Williges, 
et  al.  (1978)  using  a  double  cross-validation  procedure.  Study  II 
experimentally  tests  the  usefulness  of  these  regression  equations  as  a 
decision  rule  for  assigning  individuals  to  a  training  strategy. 


STUDY  I 


Introduction 

This  study  began  by  testing  the  validity  of  the  regression  equations 
presented  by  Savage,  et  al.  (1978)  as  a  basis  for  a  decision  rule.  The 
cross-validation  was  performed  by  obtaining  a  second  sample  of  subjects 
find  correlating  the  predicted  scores  from  the  equations  with  the  actual 
scores.  A  double  cross-validation  procedure  was  conducted  by  generating 
new  regression  equations  from  the  second  sample.  The  new  equations  were 
then  used  to  predict  back  to  the  original  sample  of  Savage,  et  al.  (1978). 
The  validity  of  these  equations  was  determined  by  correlating  the 
predicted  scores  from  the  new  equations  with  the  actual  scores  from  the 
original  sample.  If  the  correlations  appear  to  be  similar  (indicating 
that  the  regression  equations  are  good  predictors  of  time  to  exit)  and 
the  predictor  variables  are  relatively  consistent  between  the  original 
and  new  equations,  then  the  two  samples  could  be  pooled  to  produce  more 
stable  regression  equations  (Kerlinger  and  Pedhazur,  1972).  These 
validation  procedures  were  used  both  for  the  regression  equations 
predicting  training  performance  (Savage,  et  al.,  1978)  and  the 
regression  equations  predicting  transfer  performance  (Williges,  et  al., 
1978) . 

Method 

Subjects.  Ten  male  and  ten  female  subjects  were  randomly  assigned 
to  each  training  condition  (adaptive  or  fixed)  for  a  total  of  40 


subjects.  All  subjects  were  volunteers  and  were  paid  for  their 

participation  in  the  experiment.  In  addition,  all  subjects  were 

•  •  . 

nonpilots,  right-handed,  and  naive  about  the  experiment. 

Tests.  The  same  pretest  battery  used  by  Savage,  et  al.  (1978) 
and  Williges,  et  al.  (1978)  was  given  to  each  subject.  Table  3 
summarizes  the  six  pretests  and  their  respective  reliability  correla¬ 
tions  . 

The  first  test  consisted  of  six  30-second  trials  on  a  Lafayette 
Instrument  Company  pursuit  rotor.  The  turntable  was  25  cm  in  diameter 
and  the  target  was  approximately  2  cm  in  diameter.  The  pursuit  rotor 
measures  general  pursuit  tracking  ability.  A  previously  conducted 
pilot  study  found  performance  on  the  pursuit  rotor  correlating  -0.76 
with  adaptive  training  and  -0.71  with  fixed  training  (n  *  10,  all 
males)  . 

The  second  test  was  the  Embedded  Figures  Test  (Witkin,  Oltman, 

Raskin,  and  Karp,  1971) .  The  Embedded  Figures  Test  (EFT)  measures 
the  perceptual  ability  of  field  independence  and  field  dependence. 

More  specifically,  the  EFT  assesses  the  ability  to  break  up  an  organized 
visual  field  in  order  to  keep  a  part  of  it  separate  from  that  field. 

The  better  the  performance  on  the  EFT,  the  more  field  independent  one 
is.  Field  independence  appears  to  be  important  in  motor  skill  learning 
in  that  it  correlates  with  driving  behavior  (Goodenough,  1976),  piloting 
(Long,  1972),  and  tracking  (Benfari  and  Vitale,  1965).  Field  indepen¬ 
dence  also  correlates  with  arousal  level  (Oltman,  1964) ,  which 
Marteniuk  (1976)  states  is  necessary  for  selective  attention.  Furthermore, 
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TABLE  3 

Tests  Used  as  Predictor  Variables 


Test:  Reliability 

Embedded  Figures  Test  (perceptual  style  of  field  independence)  .  0.80 

Map  Memory  (visual  memory)  .  C.77 

Identical  Pictures  (perceptual  speed)  .  0.84 

Maze  Tracing  (spatial  scanning)  .  0.90 

Cube  Comparison  (spatial  orientation)  .  0.77 

Pursuit  Rotor  (continuous  motor  skill-tracking)  .  ** 

I/E  Scale  (introversion-extroversion)*  .  0.72 

TV  Handball  (continuous  motor  skill-tracking)  .  ** 


*The  I/E  Scale  and  the  TV  Handball  tests  were  not  used  in  the 
validation  study  or  in  the  Savage,  et  al.  (1978)  study. 

**The  reliability  coefficient  is  not  known. 


Cratty  (1967)  points  out  that  an  analytical  perceptual  style  such  as 
field  independence  is  important  for  early  stages  of  motor  skill 
learning.  Considerable  interest  in  the  relationship  between  field 
independence  and  motor  skills  is  evident.  Much  of  this  interest  is 
based  on  the  fact  that  field  independence  has  correlated  with  personality 
factors  such  as  creativity,  social  orientation,  individuality,  and 
impulsivity  and  with  cognitive  factors  such  as  memory  and  problem¬ 
solving  behavior.  Therefore,  performance  on  the  EFT  seems  to  be  an 
important  factor  in  motor  skill  learning. 

The  next  four  tests  were  paper-and-pencil  tests  from  the  Ekstrom, 
French,  Harman,  and  Dermen  (1976)  battery.  The  Map  Memory  Test  is  a 
test  of  visual  memory.  It  assesses  the  ability  to  remember  the 
configuration,  location,  and  orientation  of  figural  material.  There 
is  evidence  that  the  coding  of  motor  information  in  memory  is  visual 
and  spatial  in  nature.  Marteniuk  (1976)  states  that  memory  is  necessary 
in  continuous  motor  tasks  such  as  tracking.  Furthermore,  Marteniuk 
points  out  that  memory  is  a  factor  in  the  various  mechanisms  involved 
in  motor  performance  such  as  the  perceptual  mechanism  (the  detection 
of  and  attention  to  information  input) ,  the  decision  mechanism  (the 
process  of  how  to  respond  to  the  information  input) ,  and  the  effector 
mechanism  (the  organization  of  a  response).  Keele  (1973),  Laabs  (1973), 
Marteniuk  and  Roy  (1972) ,  and  Posner  (1967)  report  that  it  appears  to 
be  a  large  number  of  individual  differences  in  memory  capacity,  particu¬ 
larly  the  coding  of  information.  Finally,  Adams  and  Dijkstra  (1966) , 
Bilodeau,  Sulzer,  and  Levy  (1962),  and  Posner  (1967)  postulate  that  an 
"image"  may  be  the  form  that  movement- related  information  takes  in 


short-term  memory.  Therefore,  the  Map  Memory  Test  was  chosen  to  tap 

short-term  visual  memory  capacity  by  measuring  how  well  the  individual 

.  • . 

can  remember  visual  information. 

The  Identical  Pictures  Test  is  a  test  of  perceptual  speed.  This 
test  assesses  speed  in  comparing  figures  or  symbols,  or  carrying  out 
other  very  simple  tasks  involving  visual  perception  such  as  how  rapidly 
an  individual  can  process  perceptual  information.  Fleishman  and 
Hempel  (1954,  1956)  found  this  factor  important  in  motor  skill  learning 
Marteniuk  (1976)  discussed  the  necessity  of  an  individual  to  detect 
rapidly  and  to  filter  out  perceptually  irrelevant  information  in  motor 
learning.  Other  aspects  of  perceptual  speed  involve  selective 
attention,  scanning  and  searching,  and  pattern  recognition.  Thus,  this 
test  assesses  an  individual's  ability  to  handle  and  reduce  perceptual 
information . 

The  Cube  Comparisons  Test  measures  spatial  orientation.  It 
assesses  the  ability  to  perceive  spatial  patterns  or  to  maintain 
orientation  with  respect  to  objects  in  space.  This  test  measures  how 
well  an  individual  can  maintain  a  clear  spatial  perspective  of  objects 
in  the  environment.  Cratty  (1967)  and  Marteniuk  (1976)  pointed  out 
that  the  coding  of  motor  information  appears  to  be  spatial  as  well  as 
visual.  Fleishman  and  Rich  (1963)  have  demonstrated  that  subjects  who 
were  best  in  spatial  sensitivity  performed  better,  early  in  learning, 
than  those  who  were  worst  in  spatial  sensitivity.  This  study  again 
demonstrates  that  spatial-visual  information  is  important  in  motor 


learning. 
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The  Maze  Tracking  Test  is  a  test  of  spatial  scanning.  This  test 

assesses  the  speed  in  exploring  visually  a  wide  or  complicated  spatial 

•  _ 

field  and  measures  how  well  an  individual  can  visually  scan  a  field 
quickly  for  the  correct  path.  This  scanning  behavior  is  often  called 
visual  pursuit.  The  logic  for  this  test  is  basically  the  same  as  for 
spatial  orientation.  The  specific  task,  maze  tracing,  is  similar  to 
a  pursuit  tracking  task  which  could  indicate  a  good  predictor  of 
pursuit  tracking  performance. 

Training  conditions .  A  two-dimensional  pursuit  tracking  task  was 
used  in  the  training  task  involving  two  possible  training  approaches: 
fixed  difficulty  and  adaptive.  The  fixed  difficulty  training  condition 
uses  the  traditional  approach  to  motor  skill  training  in  which  the 
trainees  are  presented  the  criterion  task  initially  and  their  error 
decreases  as  training  progresses.  This  approach  does  not  allow  for 
individual  differences  which  often  results  in  the  task  being  too 
difficult  or  too  easy  for  the  trainee. 

The  adaptive  training  condition  involves  an  adaptive  logic  which 
manipulates  task  difficulty  in  a  closed-loop  system  (Kelley,  1969) .  In 
a  closed-loop  system,  feedback  from  the  output  of  the  former  system  is 
present  and  can  be  used  to  modify  subsequent  outputs.  Therefore,  student 
performance  is  monitored  and  compared  to  a  standard.  The  system  adjusts 
the  difficulty  of  the  training  task  so  that  the  performance  of  the 
student  is  relatively  stable  throughout  training.  The  result  is  that 
the  adaptive  training  condition  adjusts  to  individual  skills  through  the 
feedback  loop.  The  adaptive  logic  in  this  task  used  a  total  of  1,851  task 


difficulty  steps  requiring  a  minimum  time  of  111.1  seconds  to  reach  the 

exit  criterion  level  of  difficulty. 

.  • . 

Equipment.  The  training  and  transfer  tasks  were  generated  using 
a  laboratory-developed  software  package  run  on  a  Digital  Equipment 
Corporation  POP  11/55  digital  computer  which  is  linked  to  a  Tektronix 
4014-1  computer  display  terminal  and  a  Measurement  System  Model  435 
two-axis  pressure  control  stick.  The  software  responsible  for  generating 
the  tracking  task  was  developed  around  two  independent  real-time  cycles: 
a  60-Hz  cycle  to  refresh  the  display  and  a  60-ms  cycle  to  calculate  and 
update  the  task.  (See  Appendix  I  for  a  more  detailed  description  of  the 
equipment  and  the  software,  and  see  Appendix  II  for  the  software  source 
listing. ) 

Procedure.  Each  subject  completed  the  battery  of  six  pretests 
to  acquire  scores  for  the  predictor  variables.  First,  the  subject 
tracked  on  the  pursuit  rotor  to  six  30-second  trials  with  a  10-second 
rest  period  between  each  trial.  Each  trial  began  with  the  stylus  resting 
on  the  target.  The  subject  was  to  keep  the  stylus  on  the  target  while 
the  turntable  rotated  in  a  clockwise  motion  at  6.005  radians/second.  The 
mean  time  on  target  across  the  six  trials  was  the  score  for  this  test. 

Next,  the  EFT  was  administered  and  was  contained  on  7.6  cm  by 
12.7  cm  cards.  Each  subject  had  one  practice  trial  which  consisted  of 
viewing  a  complex  figure  for  15  seconds  after  which  a  simple  figure  was 
viewed  for  10  seconds.  Finally,  the  subject  was  shown  the  complex 
figure  again  and  was  instructed  to  locate  the  embedded  simple  figure. 
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The  sum  of  the  times  required  to  find  the  simple  figure  for  the  12 
trials  was  the  score  for  the  EFT. 

.  •  ■ 

The  next  four  tests  were  paper-and-pencil  tests  enclosed  in  three 
to  six  page  booklets.  The  first  page  of  each  test  contained  instructions 
and  practice  items,  and  each  test  had  two  parts.  The  score  for  each 
test  was  the  average  of  the  two  parts.  Each  part  of  each  test  was 
scored  by  subtracting  the  number  of  incorrect  items  from  the  number  of 
correct  items.  The  Cube  Comparison  Test  contained  21  pairs  of  cubes 
and  a  three  minute  time  limit  for  each  part.  The  subject's  task  was  to 
indicate  which  items  present  drawings  that  can  be  of  the  same  cube  and 
which  items  present  drawings  of  different  cubes.  For  the  Map  Memory 
Test,  each  part  consisted  of  a  study  page  and  a  test  page  with  three 
minutes  for  studying  and  three  minutes  for  testing.  Both  parts 
contained  12  maps.  The  task  was  to  identify  maps  which  were  presented 
on  the  previous  study  page.  Each  part  of  the  Identical  Pictures  Test 
consisted  of  48  items  with  one  and  one-half  minutes  allowed  for  each 
part.  The  subject's  task  was  to  identify  which  one  of  five  pictures  was 
identical  to  a  standard  figure.  Finally,  the  Maze  Tracing  Test  contained 
24  mazes  and  a  three  minute  time  limit  for  each  part.  The  task  was  to 
find  and  mark  an  open  path  through  the  series  of  mazes. 

Following  the  completion  of  the  six  pretests,  each  subject  learned 
a  pursuit  tracking  training  task  illustrated  in  Figure  1.  The  training 
task  involved  two  independent,  random,  band- limited  functions  which 
determined  the  X-Y  coordinates  of  the  forcing  function  symbol  ("X")  on 
the  display.  The  boundaries  of  the  symbol  movement  are  determined  by 
the  band-limited  function.  The  control  output  ("0”)  was  generated  from 


Flyura  l.  Tlia  pursuit  tracking  task 


inputs  from  the  isometric  controller.  The  symbol  size  was  a  0.64  *  0.48  cm 


rectangle  on  the  Tektronix  screen.  The  tracking  area  on  the  screen  was 

.  *  - 

18  cm  x  18  cm.  Feedback  bars  appeared  outside  this  area.  In  Figure  1, 
the  left  feedback  bar  displays  tracking  accuracy  with  the  tolerance 
limits  defined  by  the  horizontal  lines  at  the  top  of  the  bar.  The  right 
feedback  bar  displays  the  level  of  difficulty  of  the  task  with  the  top 
of  the  bar  being  the  exit  criterion.  In  order  to  exit  from  the  task, 
subjects  had  to  maintain  the  exit  criterion  level  of  difficulty  and 
an  acceptable  tracking  accuracy  for  a  period  of  20  seconds.  The  viewing 
distance  from  the  screen  was  1  m.  A  forehead  rest  was  used  to  maintain 
a  constant  viewing  distance. 

The  control  system  dynamics  were  neither  pure  rate  nor  pure 
acceleration.  With  pure  acceleration  dynamics,  the  task  would  have 
involved  an  unreasonable  amount  of  time  to  learn  the  task,  while  with 
pure  rate  dynamics,  the  task  would  have  been  too  simple  for  most  subjects. 
The  order  of  the  control  system  was  1  +  a,  where  a  is  the  proportion 
of  acceleration  control.  The  positions  of  the  "O"  were  calculated  using 
both  pure  rate  and  pure  acceleration  dynamics  for  each  control  stick 
output.  The  following  transfer  function  describes  the  system  dynamics 
in  each  axis: 


eox(s) 


(1  -  a)K 


1  s 


0*1*2 


®iX 


(1) 


where 


0ox(S)  *  the  Laplace  transform  of  the  scalor  change  in  position  of 
the  "0"  in  one  axis, 

a  *  a  weighting  constant  used  to  weight  rate  control  and 
acceleration  control  on  a. relative  basis, 
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■  gain  constant. 

K  ■  time  constant, 

0. ^  -  input  force  on  the  control  stick  in  one  axis,  and 
S  «  the  Laplace  transform  independent  variable. 

In  the  present  study,  a  was  equal  to  0.80.  The  constants  and  K 2 
were  derived  as  a  function  of  the  maximum  allowable  speed  of  the  "O" 
(50.8  cm/s)  and  the  tracking  area  on  the  display  (see  Appendix  I) .  In 
equation  (1),  was  equal  to  1.813^  cm/s/N  and  K 2  =  1.4285713  s 

Task  difficulty  was  manipulated  in  terms  of  the  movement  of  the 
forcing  function  symbol  ("X")  and  was  generated  by  the  minicomputer  by 
simulating  a  computer-operated  control  stick.  The  simulated  control 
stick  output  would  be  0^x  in  equation  (1) .  The  constants  and 
are  again  derived  as  a  function  of  the  maximum  allowable  speed  of  the 
"X"  (20.3  cm/s),  the  minimum  speed  of  the  "X"  (0.025  cm/s),  and  the 
tracking  area  on  the  display  (see  Appendix  I) .  The  constant  K ^  was 
0.7246  cm/s/N  and  K ^  was  equal  to  0.571*  s  .  A  random  number  generator 
internal  to  the  minicomputer  simulated  the  input  of  the  forcing 
function. 

The  adaptive  logic  used  1851  task  difficulty  steps  which  required 
111.1  s  to  reach  exit  criterion.  The  criterion  level  of  task  difficulty 
was  the  maximum  movement  speed  of  the  forcing  function  symbol  ("X") : 

20.3  cm/s.  The  tolerance  limit  was  10%  of  the  screen  diagonal  (2.5  cm). 
Absolute  tracking  error  was  computed  every  60  ms. 

A  maximum  of  15  three-minute  trials  alternating  with  one-minute 
'«•*-  periods  were  given  to  avoid  possible  fatigue  effects.  There  was  a 
•minute  rest  between  the  training  and  transfer  tasks. 


Tha  transfer  task  was  similar  to  the  training  task  except  that  no 

performance  information  {feedback  bars)  was  provided.  The  transfer 

.  * . 

task  lasted  six  minutes  during  which  three  levels  of  difficulty  in 
terms  of  the  maximum  speed  of  the  forcing  function  changed  each  minute. 

The  three  levels  of  difficulty  consisted  of  the  exit  criterion  (20.3  cm/s, 
level  of  difficulty  2),  more  difficult  than  the  exit  criterion  (30.5  cm/s, 
level  of  difficulty  3) ,  and  less  difficulty  than  the  exit  criterion 
(10.2  cm/s,  level  of  difficulty  1).  The  order  to  task  difficulties 
presented  was  the  same  for  each  subject:  levels  of  difficulty  2,  1,  3, 

1,  2,  3. 

Results  and  Discussion 

Training  data.  A  double  cross-validation  procedure  was  employed 

where  the  original  equations  were  used  to  predict  time  to  exit 

2 

performance  for  the  new  sample  (R.v,) .  and  new  regression  equations 
were  generated  from  the  new  sample  and  used  to  predict  time  to  exit 
performance  for  the  first  sample  (/?■>  )  .  For  both  samples,  correlations 

were  calculated  between  the  predicted  and  the  actual  scores  and 

■> 

R^)  .  These  correlations  are  presented  in  Table  4. 

«•  «i» 

It  was  expected  that  a  reduction  or  shrinkage  would  occur  in  the 
cross-validation  correlations  when  compared  to  the  original  correlations 
due  to  the  new  samples  of  subjects  and  different  testing  times.  When 
the  regression  equations  from  the  first  sample  were  used  to  predict 
time  to  exit  performance  for  the  second  sample,  the  cross-validated 

A 

squared  correlations  (R^,,)  were  similar  to  the  estimates  of  shrinkage 

(r"  ).  However,  the  regression  equations  for  the  adaptive  female  and 
1 
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TABLE  4 

Coefficients  of  Multiple  Determination  from  Cross-Validation 


Overall 

Male 

Female 

Adaptive  Training: 

R~ 

11 

0.721 

0.750 

0.817 

R2 

0.690 

0.717 

0.756 

S1 

*L 

0.832 

0.619 

0.306 

2 

*22 

0.859 

0.841 

0.827 

R2 

0.833 

0.796 

0.741 

S2 

2 

R-  <* 

21 

0.699 

0.578 

0.511 

t 

Fixed  Difficulty  Training: 

rH 

*11 

0.639 

0.803 

0.431 

4 

0.610 

0.758 

0.374 

4  ' 

0.444 

0.178 

0.482 

4 

0.611 

0.474 

0.905 

R2 

S2 

0.538 

0.324 

0.856 

R2 

21 

0.472 

0.333 

0.003 

NOTE:  R~  *  variance  accounted  for  of  regression  equation,  original  sample; 

2  11  2 
Rs^  ■  estimate  of  shrinkage,  original  sample;  R£2  *  squared  correlation  of 

predicted  scores  with  actual  scores  of  new  sample;  R^  ■  variance  accounted 

2  22 

for  of  regression  equation,  new  sample;  Rs.  *  estimate  of  shrinkage,  new 
2  i 

sample;  R^  *  squared  correlation  of  predicted  scores  with  actual  scores  of 
original  sample. 
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fixed  male  subjects  did  not  produce  correlations  similar  to  the  estimate 

of  shrinkage.  When  the  new  regression  equations  generated  from  the 

.  ». 

second  sample  were  used  to  predict  time  to  exit  performance  for  the 

2 

first  sample,  the  cross-validated  squared  correlations  (R~^)  were  again 

2 

similar  to  the  estimates  of  shrinkage  (R  ) .  The  exception  was  the 

s2 

regression  equation  developed  for  the  fixed  female  subjects.  Although 
the  cross-validated  squared  correlation  for  the  fixed  male  subjects  was 

similar  to  the  estimate  of  shrinkage,  the  multiple  squared  correlation 

2 

(R^)  WaS  relativelV  l°w  initially. 

Several  aspects  of  these  data  should  be  noted.  First,  the  multiple 
squared  correlations  and  the  cross-validated  squared  correlations  were 
consistently  high  for  the  overall  regression  equation.  Furthermore, 
when  comparing  the  regression  equations  derived  from  the  first 
sample  and  the  second  sample,  the  predictor  variables  were  consistent. 
The  only  exception  was  the  addition  of  a  new  predictor  variable  for 
the  overall  fixed  training  regression  equation;  however,  it  was  the 
least  weighted  variable.  The  regression  equations  developed  separately 
for  male  and  female  subjects  were  not  consistent  in  the  cross-validation 
procedure,  and  the  predictor  variables  had  a  tendency  to  change  in  the 
regression  equation  from  the  second  sample  when  compared  with  the 
regression  equation  from  the  first  sample. 

Given  these  results ,  the  two  samples  were  combined  and  an  overall 
regression  equation  was  calculated  for  both  training  models.  These 
combined  overall  regression  equations  are  shown  in  Table  5.  Each  of 
these  regression  equations  predicting  time  to  exit  performance  is 
reliable  at  the  0.0001  level  of  significance.  Both  regression  equations 
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TABLE  5 

Combined  Sample  Regression  Equations  for  Training  Time  to  Exit 
Adaptive: 

TE  -  1326.85  +  381.82  EF  -  307.48  MM  +  259.52  SE 

n  «  51 

R2  *  0.756 

R2  -  0.740 
s 

p  £  0.0001 
Fixed  Difficulty : 

TE  »  994.57  +  405.77  EF  +  251.3  IP  -  139.28  CC 

n  *  48 

R2  =  0.632 

R2  *  0.607 
s 

p  £  0.0001 

CC  *  Cube  Comparison  Test,  number  correct  minus  number  incorrect 
EF  =  Embedded  Figures  Test,  time  to  locate  simple  figure 
IP  *  Identical  Pictures  Test,  number  correct  minus  number  incorrect 
MM  »  Map  Memory  Test,  number  correct  minus  number  incorrect 


SE  *  Sex 
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have  the  same  dominant  predictor  variable:  the  Embedded  Figures  Test. 

All  other  predictor  variables  are  different.  Sex  is  a  reliable 

.  • . 

predictor  variable  only  in  the  regression  equation  for  adapative  training. 
These  differences  in  the  regression  equations  should  be  sufficient  to 
discriminate  time  to  exit  performance  between  the  two  training  strategies 
to  serve  as  a  decision  rule  to  determine  a  training  strategy  for  the 
individual . 

Transfer  data.  Table  6  presents  the  double  cross-validation  data 

for  the  regression  equations  predicting  mean  RMS  vector  error  in  the 

transfer  task.  These  data  did  not  cross-validate  in  that  the 

2  2 

cross-validated  squared  correlations  and  r2i^  ^ail  to  compare  with 

2  2 

the  estimates  of  shrinkage  and  R^  ) .  There  were  three  statistically 

significant  correlations :  the  overall  regression  equation  for  adaptive 
training,  r21  »  0.370,  p  <  0.05,  and  the  overall  regression  equations 
for  fixed  training,  r12  “  0-546,  p  <  0.01,  and  r  =  0.464,  p  <  0.05. 
Because  of  these  statistically  significant  correlations,  the  two  samples 
were  combined  to  develop  an  overall  regression  equation  for  each  training 
model.  These  regression  equations  appear  in  Table  7.  Both  regression 
equations  are  reliable  at  the  0.0001  level  of  significance.  As  was  found 
with  the  regression  equations  predicting  time  to  exit  performance,  the 
Embedded  Figures  Test  is  the  dominant  predictor  variable.  The  only 
predictor  variable  not  common  to  both  regression  equations  is  the  Cube 
Comparison  Test  in  fixed  training. 

The  predictor  variable  in  the  combined  overall  regression  equations 
presented  in  Table  7  were  all  contained  in  either  the  regression  equation 
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TABLE  6 

Coefficients  of  Multiple  Determination  from  Cross-Validation 


Overall 

Male 

Female 

Adaptive  Training: 

11 

0.691 

0.545 

0.906 

R2 

0.629 

0.484 

0.875 

S1 

R2 

12 

0.027 

0.036 

0.124 

*2 

22 

0.474 

0.383 

0.824 

R2 

0.412 

0.306 

0.736 

S2 

4 

0.137* 

0.046 

0.001 

Fixed  Difficulty  Training: 

_2 

Rii 

0.454 

0.807 

R2 

0.408 

0.697 

S1 

R^ 

12 

0.298** 

0.191 

r\ 

22 

0.507 

0.642 

0.369 

R2 

0.415 

0.597 

0.290 

S2 

R.2 

21 

0.215* 

0.043 

0.109 

*p  <  0.05.  **p  <  0.01. 

NOTE:  For  definition  of  terms,  see  Table  4. 


[ 

l 

I 


3.1 


TABLE  7 

Combined  Sample  Regression  Equations  for  Predicting  Transfer  Performance 
Adaptive  Training: 

RMS  -  0.1088  +  0.0123  EF  +  (-0.0071)  MM  +  0.0062  SE 
R2  *  0.445 

R2  -  0.410 

sr 

n  *  51 

p  =  0.0001 

Fixed  Difficulty  Training: 

RMS  -  0.1078  +  0.0086  EF  +  0.0068  SE  +  0.0067  CC  +  (-0.0042)  MM 

R2  *  0.462 

R2  =  0.412 
s 

n  *  48 
p  =■  0.001 

EF  =>  Embedded  Figures  Test,  time  to  locate  simple  figure 
MM  *  Map  Memory  Test,  number  correct  minus  number  incorrect 
CC  *  Cube  Comparison  Test,  number  correct  minus  number  incorrect 


SE  *  Sex 


developed  from  the  first  sample  or  the  regression  equation  developed 

from  the  second  sample.  The  two  regression  equations  calculated  from 

.  «. 

the  two  samples  separately  were  very  dissimilar.  Therefore,  these 
regression  equations  predicting  mean  RMS  vector  error  in  the  transfer 
task  need  to  be  validated  with  a  second  sample  of  subjects. 


STUDY  II 


Introduction 

The  purpose  of  the  second  study  was  to  test  the  usefulness  of 
the  regression  equations  as  a  decision  rule;  that  is,  can  one  optimize 
the  training  of  an  individual  by  using  predicted  scores  from  the 
regression  equations  to  determine  the  best  training  condition  for  that 
individual?  To  determine  the  usefulness  of  these  equations,  subjects 
were  either  matched  or  mismatched,  according  to  their  predicted  time 
to  exit  scores,  to  a  training  strategy.  If  a  significant  difference 
exists  between  the  matched  and  mismatched  subjects ,  with  the  matched 
subjects  requiring  less  time  to  train,  then  it  would  appear  that  the 
equations  are  useful  for  assigning  subjects  to  an  optimal  training 
strategy  when  subjects  are  mismatched. 

A  third  treatment  (aside  from  matched  and  mismatched)  was  a  random 
assignment  condition  in  which  ,  ub jects  were  randomly  assigned  to  a 
training  strategy.  A  significant  difference  between  the  matched  condition 
and  the  random  condition,  with  the  matched  subjects  exhibiting  superior 
performance,  would  not  only  demonstrate  that  subjects  are  properly 
matched  to  their  respective  training  condition,  but  it  would  be  an 
indication  of  the  predictive  power  of  the  regression  equations,  in  that 
approximately  50%  of  the  subjects  in  the  random  condition  would  be  assumed 
to  be  properly  matched  to  a  training  condition  initially. 

This  experiment  also  served  as  a  second  validation  of  the  existing 
regression  equations  predicting  training  performance  and  the  validation 
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of  the  regression  equations  predicting  transfer  performance.  As  in 

Study  I,  the  predicted  scores  will  be  correlated  with  the  actual 

•  • . 

scores. 

Method 

Experimental  design  and  analysis.  Figure  2  gives  a  pictorial 
representation  of  the  2*2X3  factorial  experimental  design  for  the 
training  task.  The  variables  are  training  (adaptive  and  fixed) ,  sex 
(male  and  female),  and  assignment  (matched,  mismatched,  and  random). 

All  of  the  variables  are  between-subject  effects  and  an  analysis  of 
variance  (ANOVA)  was  used  to  analyze  the  data.  The  experimental  design 
for  the  transfer  task  included  the  additional  within-subject  variable 
of  level  of  difficulty.  The  Newman-Keuls  multiple  comparison  test  was 
used  to  analyze  all  significant  effects. 

The  present  experiment  served  as  a  second  validation  of  the 
regression  equations  predicting  both  training  and  transfer  performance 
(see  Tables  5  and  7) .  The  validation  analysis  consisted  of  correlating 
the  predicted  scores  with  the  actual  scores.  Finally,  new  regression 
equations  predicting  mean  RMS  vector  error  in  the  transfer  task  were 
generated  using  the  two  additional  tests.  This  analysis  used  five 
stepwise  procedures  from  the  SAS  package  (see  Barr,  Goodnight,  Sail, 
and  Helwig,  1976) . 

Materials  and  procedure.  The  equipment,  training  conditions,  and 
tests  and  their  procedures  were  the  same  as  in  the  first  study.  In 
addition  to  the  six  tests  described  in  Study  I,  two  additional, 
supplemental  tests  were  administered.  The  I/E  Scale  (Rotter,  1954,  1966) 


Figure  2.  The  experimental  design 


* 


36 

involves  subjects  answering  questions  about  situations  of  interest  in 

our  society  over  which  they  may  be  perceived  to  have  control  or  over 

•  •  . 

which  they  perceive  external  factors  to  have  control.  The  answers  to 
these  questions  consisted  of  choosing  which  statement  they  believed  to 
be  more  true.  Ryckman  (1978)  found  the  1/E  Scale  to  correlate  with 
tracking  performance  on  a  pursuit  rotor. 

The  second  additional  test  was  a  TV  handball  game  which  consisted 
of  maximizing  the  time  in  the  game.  The  task  involved  hitting  a  ball 
with  a  paddle  which  the  subject  controlled  in  a  vertical  dimension. 

The  more  often  the  subject  hits  the  ball,  the  longer  he  stays  in  the 
game.  Once  the  subject  missed  the  ball  15  times,  the  game  ended  and 
the  time  was  recorded.  The  game  was  displayed  on  a  48.3  cm  television 
screen.  A  pilot  study  found  this  motor  task  to  be  correlated  with  mean 
transfer  performance  (RMS  vector  error) :  r  =  -0.642  for  adaptive 
training  and  0.422  for  fixed  training  ( n  =  10,  all  males). 

The  addition  of  the  I/E  Scale  assesses  one  aspect  of  the  personality 
domain  of  the  individual  which  follows  the  suggestion  of  Conway  and 
Norman  (1974) ,  whereas  the  addition  of  the  TV  handball  could  be  considered 
as  a  task-specific  factor  which  follows  the  suggestion  of  Williges,  et  al . 
(1978) .  These  new  tests  will  be  used  with  the  other  six  tests  to  develop 
new  regression  equations  for  predicting  transfer  performance  in  an  attempt 
to  account  for  more  variance  than  the  present  regression  equations 
presented  in  Table  7. 

Assignment  of  the  subjects.  The  critical  aspect  of  this  experiment 


involved  the  assignment  of  the  subjects  to  a  training  condition  using 
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the  predicted  time  to  exit  scores  of  the  regression  equations  presented 

in  Table  5.  The  first  step  in  the  assignment  was  to  randomly  assign 

•  «  . 

20  male  and  20  female  subjects  to  the  matched,  mismatched,  or  random 
condition.  If  the  subject  were  in  the  random  condition,  then  he/she 
would  be  randomly  assigned  to  one  of  the  two  training  conditions. 

However,  the  predicted  time  to  exit  scores  from  the  regression  equations 
in  Table  5  determined  which  training  condition  the  subject  received  in 
both  the  matched  and  mismatched  conditions. 

Results  and  Discussion 

Training  data.  The  results  of  the  ANOVA  on  the  time  to  exit  scores 
(measured  in  seconds)  found  two  significant  effects:  sex  and  assignment. 
The  cell  means  are  presented  in  Table  8  and  the  ANOVA  summary  table  is 
presented  in  Table  9.  For  the  main  effect  of  sex,  male  subjects 
required  significantly  less  time  to  exit  than  female  subjects.  The 
mean  time  to  exit  for  male  subjects  was  685.42  s;  whereas,  for  female 
subjects  the  mean  time  to  exit  was  1306.09  s.  The  Newman-Keuls  test 
found  that  for  the  assignment  main  effect,  the  matched  subjects  required 
significantly  less  time  to  exit  than  either  the  random  or  the  mismatched 
subjects  at  the  0.01  level.  The  difference  between  the  random  and 
mismatched  subjects  was  not  reliable.  The  mean  time  to  exit  was  599.38  s 
for  the  matched  subjects,  1122.08  s  for  the  random  subjects,  and 
1265.80  s  for  the  mismatched  subjects. 

The  results  strongly  support  the  use  of  these  regression  equations 
as  a  decision  rule  which  selects  a  training  strategy  for  the  individual. 
When  subjects  are  matched  to  a  training  strategy,  a  savings  of  47%  in 
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TABLE  8 


Cell  Means 

for  the  ANOVA 

on  the  Time 

to  Exit  Data 

Adaptive  Training 

Fixed  Training 

Mean 

Male 

Female 

Male 

Female 

Matched 

324.97 

834.37 

430.70 

807 . 50 

599.38 

Random 

694.96 

1633.39 

937.59 

1222.37 

1122.08 

Mismatched 

1013.00 

1814.51 

711.28 

1524.41 

1265.80 

Mean. 

677.64 

1427.42 

693.19 

1184.77 

TABLE  9 


ANOVA  Source  Table 

for  the 

Time  to  Exit  Data 

Source 

df 

MS 

F 

P 

Training  (T) 

1 

386864. 852 

1.36 

<  0.2464 

Assignment  (A) 

2 

4919876.150 

17.27 

<  0.0001 

Sex  (S) 

1 

11557148.496 

40.57 

<  0.0001 

T  *  A 

2 

287595.476 

1.01 

<  0.3678 

T  x  S 

1 

500037.867 

1.76 

<  0.1880 

A  x  S 

2 

332250.314 

1.17 

<  0.3154 

T  x  A  x  s 

2 

306203.480 

1.07 

<  0.3449 

Subjects/TAS 

108 

284842.349 
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training  time  can  result  when  compared  to  subjects  who  are  randomly 

assigned  and  a  savings  of  53%  when  compared  to  subjects  who  are 

•  *  . 

mismatched.  Therefore,  when  the  goal  of  designing  a  training 
program  is  to  maximize  savings  in  training  time,  the  trainer  would 
profit  by  using  a  similar  methodology  of  matching  subjects  with  a 
training  strategy. 

Theoretically,  the  random  condition  should  have  50%  of  the 
subjects  matched  and  50%  mismatched.  Zn  this  study,  the  random 
condition  resulted  in  22  subjects  matched  and  18  subjects  mismatched. 
These  subjects'  data  were  then  placed  in  the  respective  assignment 
conditions  which  results  in  two  levels  of  the  assignment  variable. 

An  unequal  ns  ANOVA  was  then  conducted. 

The  results  again  found  a  significant  main  effect  of  assignment. 

The  cell  means  are  presented  in  Table  10  and  the  ANOVA  source  table 
is  presented  in  Table  11.  The  mean  time  to  exit  was  769.18  s  for  the 
matched  subjects  and  1237.96  s  for  the  mismatched  subjects.  The 
assignment  by  sex  interaction  was  also  significant.  The  Newman-Keuls 
test  found  that  the  mismatched  female  subjects  were  significantly 
different  from  the  matched  male  subjects,  the  matched  female  subjects, 
and  the  mismatched  male  subjects  at  the  0.01  level.  There  were  no 
other  reliable  differences.  The  mean  time  to  exit  for  the  mismatched 
female  subjects  was  1652.91  s;  whereas,  the  mean  time  to  exit  was 
590.96  s,  793.36  s,  and  959.27  s  for  the  matched  male  subject,  the 
mismatched  male  subjects,  and  the  matched  female  subjects,  respectively. 
The  results  of  this  interaction  are  presented  in  Figure  3. 
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TABLE  10 


Cell  Means 

for  the  Unequal 

ns  ANOVA 

on  the  Time 

to  Exit  Data 

Adaptive  Training 

Fixed 

Training 

Male 

Female 

Male 

Female 

Mean 

Matched 

520.91 

908.27 

670.36 

988.80 

769.18 

Mismatched 

882.60 

1727.99 

716.02 

1523.23 

1237.96 

TABLE  11 


Unequal  ns  ANOVA  Source  Table  for  the  Time  to  Exit  Data 


Source 

df 

MS 

F 

P 

Training  (T) 

1 

35635.63 

0.11 

<  0.7355 

Assignment  (A) 

1 

5574623.12 

17.94 

<  0.0001 

Sex  (S) 

1 

10007378.31 

32.20 

<  0.0001 

T  x  a 

1 

650364.04 

2.09 

<  0.1508 

T  x  S 

1 

20646.17 

0.07 

<  0.7971 

A  x  s 

1 

1611813.39 

5.19 

<  0.0247 

T  x  a  x  S 

1 

1699.38 

0.01 

<  0.9412 

Subjects/TAS 


112 


310754.18 
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Two  significant  results  were  found  from  the  training  data.  First, 

male  subjects  generally  required  about  half  the  time  to  learn  the  task 

.  •- 

that  the  female  subjects  required.  Second,  when  subjects  were  matched 
to  a  training  condition,  a  47  to  53%  savings  in  training  time  resulted 
when  compared  with  the  random  or  mismatched  conditions.  However,  the 
unequal  ns  ANOVA  showed  that  this  result  is  primarily  due  to  the 
mismatched  female  subjects.  There  was  no  significant  difference 
between  the  matched  and  mismatched  male  subjects  in  the  assignment  by 
sex  interaction. 

Transfer  data.  The  results  of  the  ANOVA  on  the  RMS  vector  error 
found  three  main  effects  and  one  interaction  significant.  The  cell 
means  are  presented  in  Table  12  and  the  ANOVA  summary  table  is  presented 
in  Table  13,  The  level  of  difficulty  was  found  to  be  highly  reliable 
with  the  subsequent  Newman-Keuls  test  finding  all  three  levels  of 
difficulty  reliably  different  at  the  0.01  level.  The  RMS  vector  error 
was  0.0711,  0.1160,  and  0.1486,  respectively,  for  the  lowest  level  of 
difficulty,  the  exit  criterion  of  difficulty,  and  the  highest  level  of 
difficulty.  The  effect  of  sex  was  also  highly  reliable  with  a  mean  RMS 
vector  error  of  0.1020  for  male  subjects  and  0.1218  for  female  subjects. 
The  sex  by  level  of  difficulty  interaction  was  also  found  to  be  reliable. 
The  Newman-Keuls  test  resulted  in  all  possible  combinations  being 
reliable  at  least  at  the  0.05  level.  Generally,  male  subjects  had 
smaller  error  than  female  subjects  at  each  level  of  difficulty. 

The  main  effect  of  assignment  resulted  in  a  reliable  difference. 

The  Newman-Keuls  test  found  that  matched  and  random  subjects  had  a 


TABLE  12 

Cell  Means  for  the  ANOVA  on  the  RMS  Vector  Error  Rate 


Adaptive 

Training 

Fixed 

Training 

Mean 

Male 

Female 

Male 

Female 

Matched 

0.0840 

0.0937 

0.1035 

0.1198 

0.1003 

Random 

0.0982 

0.1260 

0.0949 

0.1207 

0.1099 

Mismatched 

0.1088 

0.1349 

0.1226 

0.1359 

0.1255 

Mean 


0.0970 


0.1182 


0.1070 


0.1255 
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TABLE  13 


ANOVA  Source  Table 

for  the 

Transfer  Task  Data 

Source 

df 

MS 

F 

P 

Training  (T) 

1 

0.0066977 

2.74 

±  0 . 1008 

Assignment  (A) 

2 

0.0195094 

7.98 

£  0.0006 

Sex  (S) 

1 

0.355057 

14.53 

<  0.0002 

T  x  A 

2 

0.0055405 

2.27 

£  0.1086 

T  x  s 

1 

0.0001613 

0.07 

<  0.7977 

A  x  s 

2 

0.0014335 

0.59 

£  0.5581 

T  x  a  x  s 

2 

0.0007099 

0.29 

<  0.7485 

Subjects/TAS 

108 

0.0024444 

Level  of  Difficulty 

(L) 

2 

0.1815312 

386.22 

£  0.0001 

T  x  L 

2 

0.0003596 

0.77 

<  0.4666 

A  x  l 

4 

0.0005550 

1.82 

<  0.1262 

S  x  L 

2 

0.0024918 

5.30 

<  0.0057 

T  x  A  x  L 

4 

0.0004767 

1.01 

£  0.4009 

T  x  s  x  L 

2 

0.0006420 

1.37 

£  0.2573 

A  x  S  x  L 

4 

0.0002715 

0.58 

<  0.6792 

T  x  A  x  S  x  L 

4 

0.0001069 

0.23 

<  0.9228 

L  x  Subjects/TAS 

216 

0.0004700 
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smaller  RMS  vector  error  than  mismatched  sub jects  ( p  <_  0.01  and  p  <_  0.05, 
respectively) ,  whereas  the  matched  and  random  subjects  were  not  reliably 
different.  The  mean  RMS  vector  error  was  0.1003,  0.1100,  and  0.1255, 
respectively,  for  the  matched,  random,  and  mismatched  conditions.  This 
result  suggests  that  by  matching  subjects  to  a  training  condition  based 
on  time  to  exit  scores,  they  will  have  significantly  smaller  RMS  vector 
error  than  subjects  who  are  purposely  mismatched.  The  matched  subjects, 
however,  will  not  be  better  than  subjects  who  are  randomly  assigned. 
Therefore,  if  the  goal  of  a  training  program  is  to  maximize  transfer 
task  performance  by  matching  subjects  based  on  time  to  exit  scores,  then 
these  data  are  not  necessarily  supportive  of  this  goal  because  of  the 
lack  of  a  reliable  difference  between  the  matched  and  random  subjects. 

Given  that  22  subjects  were  matched  and  18  were  mismatched  in  the 
random  condition,  an  unequal  ns  ANOVA  was  also  performed  on  the  transfer 
task  data.  The  cell  means  are  presented  in  Table  14  and  the  ANOVA 
summary  table  is  presented  in  Table  15.  These  results  found  two  reliable 
effects:  training  and  assignment.  Subjects  trained  adaptively  had 
significantly  less  RMS  vector  error  than  subjects  trained  in  the  fixed 
condition.  The  mean  RMS  vector  error  was  0.1076  and  0.1162  for  the 
adaptive  and  fixed,  respectively.  For  the  effects  of  assignment,  matched 
subjects  had  significantly  smaller  error  in  transfer  than  mismatched 
subjects.  The  mean  RMS  vector  error  was  0.1030  and  0.1215  for  matched 
and  mismatched,  respectively.  These  two  results  suggest  that  error  can 
be  successfully  reduced  in  transfer  by  training  subjects  adaptively  or 
by  matching  subjects  to  a  training  condition.  An  8%  reduction  in  error 
was  obtained  by  the  adaptive  training  condition,  whereas  an  18%  reduction 
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TABLE  14 

Cell  Means  for  the  Unequal  ns  ANOVA  on  the  RMS  Vector  Error  Data 


Adaptive 

Training 

Fixed 

Training 

Male 

Female 

Male 

Female 

Mean 

Matched 

0.0917 

0.0957 

0.1027 

0.1175 

0.1030 

Mismatched 

0.1040 

0.1312 

0.1112 

0.1393 

0.1215 

TABLE  15 


Unequal  ns  ANOVA  Summary  Table  for  the  Transfer  of  Task  Data 


Source 

df 

MS 

F 

P 

Training  (T) 

1 

0.01249 

5.05 

<  0.0266 

Sex  (S) 

1 

0.03283 

13.27 

<  0.0004 

Assignment  (A) 

1 

0.02973 

12.01 

<  0.0005 

T  *  A 

1 

0.00165 

0.67 

<  0.4155 

T  X  S 

1 

0.00072 

0.29 

<  0.5901 

A  x  s 

1 

0.00718 

2.90 

<  0.0913 

T  x  a  x  s 

1 

0.00051 

0.21 

<  0.6495 

Subjects/TAS 

112 

0.00248 

Level  of  Difficulty 
(L) 

2 

0.17528 

373.85 

<_  0.0001 

T  x  l 

2 

0.00050 

1.06 

<  0.3465 

A  x  l 

2 

0.00125 

2.67 

<_  0.0717 

S  x  L 

2 

0.00248 

5.28 

<  0.0057 

T  x  A  x  L 

2 

0.000027 

0.06 

<  0.9447 

T  x  s  x  L 

2 

0.000820 

1.75 

<  0.1766 

A  x  s  x  l 

2 

0.000260 

0.55 

<  0.5757 

T  x  A  x  s  x  l 

2 

0.000130 

0.27 

<  0.7658 

L  x  Subject/TAS 


224 


0.000470 
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of  error  was  obtained  by  the  matched  condition.  Therefore,  if  the 

goal  of  the  trainer  is  to  maximize  transfer  performance,  then  trade-off 

.  « . 

considerations  need  to  be  considered  based  on  these  results. 

Validation  results.  This  study  also  provided  an  opportunity  to 

validate  the  cobmined  overall  regression  equations  previously  presented 

in  Tables  5  and  7.  As  in  Study  I,  the  predicted  scores  were  correlated 

with  the  actual  scores.  In  considering  first  the  regression  equations 

in  Table  5  for  predicting  time  to  exit  performance  in  adaptive  training, 

the  correlation  between  the  predicted  scores  and  the  actual  scores  was 

2 

r  =  0.789,  p  =  0.0001.  The  R  was  0.623,  which  was  somewhat  lower  than 

the  estimate  oi  shrinkage  (R2)  is  Table  5.  The  regression  equation  £or 

s 

2 

the  fixed  training  resulted  in  an  r  =  0.716,  p  =  0.0001.  The  R  was 
0.5131,  which  again  was  somewhat  lower  than  the  estimate  of  shrinkage 
(i?2)  in  Table  5. 

Additionally,  the  regression  equations  in  Table  7  predicting  mean 

RMS  vector  error  in  transfer  were  also  validated.  The  regression 

equation  for  fixed  training  resulted  in  a  near  zero  correlation  and, 

therefore,  showed  no  signs  of  being  a  valid  equation.  However,  the 

regression  equation  for  adaptive  training  resulted  in  a  r  =  0.574, 

2 

p  =*  0.0001.  The  R  was  0.330,  which  was  lower  than  the  estimate  of 

2  *■ 
shrinkage  (.R  )  in  Table  7.  It  should  be  noted  that  the  variance  accounted 
s 

for  by  this  equation  was  less  than  33%,  which  is  relatively  low  in 
comparison  to  the  equations  predicting  time  to  exit  performance. 

Given  that  the  regression  equations  predicting  transfer  performance 
were  either  not  valid  or  accounted  for  a  small  proportion  of  variance, 
new  equations  were  calculated  using  the  results  of  the  two  additional 
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tests.  These  equations  are  presented  in  Table  16.  Both  equations  are 

highly  reliable,  but  the  equation  for  fixed  training  only  accounted 

.  •- 

for  a  small  proportion  of  variance,  26%.  The  regression  equation  for 
adaptive  training  accounted  for  43%  of  the  variance  in  the  transfer 
task.  The  validity  of  this  equation  needs  to  be  determined.  In 
general,  success  in  predicting  performance  in  the  transfer  task  is 
lacking.  Fleishman  (1972)  and  Williges,  et  al.  (1978)  suggest  that 
task-specific  variables  are  evident  late  in  learning  (or  training)  and 
that  general  ability  factors  (e.g.,  information-processing  skills) 
become  less  important.  If  this  is  the  situation  in  the  present  transfer 
task,  then  task-specific  variables  are  needed  to  predict  performance. 

The  equations  in  Table  16  show  some  support  for  task-specific  variables 
in  that  both  equations  contain  a  measure  of  tracking  performance  (the 
pursuit  rotor  and  the  television  handball)  which  could  be  considered 


task-specific  variables. 
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TABLE  16 

New  Regression  Equations  Predicting  Transfer  Performance 


Adaptive 

RMS  *  0.107597  +  0.014449  EF  +  (-0.007982)  PR 
2 

R  =  0.429 
2 

R  =0.409 
s 

n  =  60 

p  =  0.0001 

Fixed 

RMS  =  0.116224  +  (-0.012541)  CC  +  (-0.008249)  TV 
2 

R  =  0.263 
2 

R  =  0.237 
s 

n  *  60 

p  =  0.0002 

EE  ■  Embedded  Figures  Test,  time  to  find  simple  figure 
PR  =  Pursuit  Rotor,  time  on  target 

CC  s  Cube  Comparison  Test,  number  correct  minus  the  number  incorrect 


TV  =  Television  Handball,  time  to  complete  game 


CONCLUSION 


The  results  of  both  studies  were  positive,  and  there  are  some 
implications  for  future  research.  Study  I  showed  that  performance  in 
the  pursuit  tracking  task  could  be  predicted  by  using  a  multiple 
regression  approach  with  information-processing  variables  as  predictors. 
In  both  training  conditions,  adaptive  and  fixed,  time  to  exit 
performance  was  predicted  reliably,  as  indicated  by  the  double  cross- 
validation  results. 

These  results  have  implications  for  future  research  on  individual 
differences  in  motor  skill  learning.  Information-processing  skills 
were  found  important  in  motor  skill  learning,  and  many  of  the  mechanisms 
of  various  information-processing  theories  dealing  with  human  performance 
could  be  tested.  Furthermore,  performance  in  other  training  models  for 
motor  skills  could  be  predicted  by  generating  new  regression  equations. 
Williges  and  Williges  (1977)  found  success  with  a  learner-centered 
training  model  for  motor  skill  training.  Other  adaptive  training  models 
could  be  utilized  with  different  adaptive  logics.  By  adding  more 
training  strategies,  a  trainer  could  more  closely  match  students  with  a 
training  strategy.  Finally,  the  prediction  of  motor  performance  may  be 
improved  by  adding  measures  of  personality  characteristics,  previous 
experience,  general  psychomotor  skills,  and  task-specific  variables. 

From  the  results  reported  here,  it  appears  that  successful  prediction  of 
transfer  performance  may  be  dependent  on  measures  of  task-specific 
variables.  This  is  particularly  true,  since  the  I/E  scale  did  not  add 
to  the  prediction  of  transfer  performance. 


Study  II  results  document  that  matching  students  to  a  training 

strategy  results  in  substantial  savings  of  training  time.  This  was 

.  •- 

particularly  true  for  female  subjects.  For  the  transfer  task,  smaller 
tracking  error  resulted  when  subjects  were  matched  as  opposed  to 
mismatched. 

Overall,  three  basic  conclusions  seem  warranted.  First,  a  methodo¬ 
logical  approach  was  utilized  by  using  regression  equations  as  a 
decision  rule.  The  results  of  this  research  show  that  this  approach 
is  successful;  however,  it  may  not  necessarily  be  the  best  approach. 
Future  research  is  needed  to  determine  the  appropriateness  of  other 
approaches,  such  as  the  use  of  discriminant  analysis  or  the  approach 
suggested  by  Conway  and  Norman  (1974).  Furthermore,  this  research 
needs  to  be  replicated,  which  is  necessary  to  determine  the  validity 
of  the  methodological  approach  used  in  this  research. 

Second,  individualized  training  is  supported  by  this  research  in 
terms  of  training  time  and  tracking  error.  In  other  situations,  such 
as  pilot  training,  individualized  training  can  be  costly,  and  certain 
trade-offs  need  consideration.  Forty-five  minutes  of  pretesting  is 
not  cost  effective  when  the  student  can  be  trained  in  30  minutes  as 
with  the  tracking  task  used  in  the  present  research.  If  the  training 
time  is  a  matter  of  days  or  weeks,  then  45  minutes  of  pretesting  can 
have  definite  benefits  for  purposes  of  matching  students  with  a  training 
strategy . 

Finally,  this  research  cam  be  thought  of  in  terms  of  searching  for 
an  "optimal"  training  strategy  for  the  individual,  optimal  used  in  the 
sense  of  matching  as  closely  as  possible  the  characteristics  of  the 
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individual  with  a  training  strategy.  Based  on  the  results  of  this 

research*  matching  trainees  with  an  "optimal"  training  strategy  will 

«  . 

• 

result  in  a  savings  of  training  time  and  better  performance  in  transfer. 
The  possibility  exists,  however,  that  an  individual  is  matched  to  a 
training  strategy,  but  it  may  not  necessarily  be  an  "optimal"  strategy. 
Therefore,  research  is  needed  to  not  only  predict  performance  within 
a  training  strategy,  but  also  to  determine  efficient  and  economic 
training  strategies  so  that  an  "optimal"  training  strategy  for  the 
individual  becomes  more  of  a  reality. 
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INTRODUCTION 


Background 

ThroughouC  the  history  of  learning  research,  there  has  been  no 
conclusive  data  to  demonstrate  that  learning  can  occur  without  the 
presence  of  some  form  of  feedback.  In  any  given  task,  feedback  is 
usually  separated  into  two  components,  intrinsic  feedback  and  aug¬ 
mented  feedback.  Intrinsic  or  fundamental  feedback  is  that  which  is 
inherent  in  a  system  and  is  encountered  in  most  daily  tasks.  For 
example,  in  driving  an  automobile,  an  increase  in  the  loudness  of  the 
engine  informs  the  operator  that  he  is  accelerating.  However,  the 
operator  can  not  precisely  tell  you  the  speed  at  any  one  point  in  the 
acceleration.  The  speedometer  provides  this  information  and  is  a 
form  of  augmented  or  supplemental  feedback  since  it  is  not  inherent 
in  the  sub-system  causing  the  automobile  to  accelerate. 

Clearly,  the  augmented  feedback  is  not  essential  to  the  system's 
operation,  but  it  is  obvious  that  the  performance  of  the  operator- 
machine  system  can  be  Improved  by  its  use.  This  is  the  basic 
difference  between  intrinsic  and  augmented  feedback— intrinsic  feedback 
is  essential  to  the  successful  operation  of  a  system,  whereas  the 
system  may  be  operated  without  the  use  of  augmented  feedback.  However, 
the  appropriate  implementation  of  augmented  feedback  into  a  system  can 
lead  to  improved  performance  of  the  operator  and,  therefore,  improve 
the  operation  of  the  system.  Consequently,  one  could  define  augmented 
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feedback  as  supplemental  Information  provided  to  the  human  operator 
which  enhances  the  fundamental  feedback  available  from  the 
operational  system. 

Although  both  forms  of  feedback  are  based  on  the  same  source  of 
information,  fundamental  feedback  is  the  more  direct  of  the  two  in 
that  the  information  in  provides  is  immediate  and  does  not  undergo  any 
transformations.  Augmented  feedback,  on  the  other  hand,  is  based  on 
a  sample  of  behavior  taken  over  time  and  is  transformed  in  the  sense 
that  the  behavior  of  the  operator-machine  system  is  compared  with  an 
external  criterion.  In  the  above  example,  the  sound  of  the  engine  is 
fundamental  feedback  because  it  is  a  direct  result  of  the  mechanical 
operations  causing  the  automobile  to  accelerate.  Since  the  speed¬ 
ometer  represents  the  past  actions  of  the  operator  and  the  information 
it  presents  is  compared  with  a  legally  or  individually  established 
criterion,  it  is  a  form  of  augmented  feedback.  Thus,  information 
supplied  by  the  augmented  feedback  aids  the  operator  to  evaluate 
performance  against  established  criteria.  The  means  by  which  augmented 
feedback  is  generated  and  the  way  in  which  it  is  utilized  by  the 
operator  indicates  that  it  must  occur  some  time  after  the  operator's 
actions  and  that  it  has  an  evaluative  function. 

Figure  1  illustrates  augmented  feedback  aiding  the  operator  in 
evaluating  performance.  The  operator  compares  system  input  (what 
should  be  done)  with  system  output  (what  actually  was  done)  and 
evaluates  the  results  of  operator  actions  against  the  external 


criterion. 


Fundamental  and  augmented  feedback  loops 
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Augmented  feedback  does  not  always  take  the  form  of  information 
presented  on  a  display.  It  could  very  well  be  comments  from  an 
instructor,  as  is  often  the  case  in  industrial  training  situations,  or 
a  paper  and  pencil  test  from  which  the  trainer  evaluates  the  trainee's 
progress  and  then  feeds  this  information  back  to  the  trainee.  Many 
training  systems  currently  employ  augmented  feedback  while  many  more 
could  benefit  from  its  use.  However,  augmented  feedback  is  of  no 
benefit  if  upon  its  withdrawal  (such  as  the  case  may  be  in  going  from 
a  job  training  situation  to  the  actual  job)  the  human  operator's 
performance  declines  to  the  point  that  he  can  no  longer  successfully 
control  the  system. 

Open-loop  versus  closed-loop  systems.  Most  learning  situations 
encountered  are  of  the  open- loop  variety.  An  individual  possessing 
a  specified  minimum  aptitude  is  expected  to  complete  successfully  a 
task  with  a  certain  level  of  difficulty  given  an  appropriate  set  of 
stimuli.  In  designing  the  learning  situation,  no  consideration  is 
usually  given  to  feedback  derived  from  the  learner's  responses.  If 
feedback  is  incorporated  into  the  system,  it  is  typically  of  the 
reinforcement  type,  indicating  performance  degradation  or  improvement, 
which  does  little  to  aid  the  learner  in  formulating  a  correct  response. 
Closed-loop  learning  systems  differ  in  that  a  reference  specifying 
the  desired  system  response  is  available  against  which  the  system's 
output  (feedback)  due  to  the  operator's  responses  are  compared.  Thus, 
the  learner  has  a  criterion  on  which  to  base  successive  responses  if 
a  discrepancy  (error)  occurs  between  the  output  of  the  system  due  to 
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a  previous  response  and  the  reference.  These  are  two  qualifying 
features  of  a  closed-loop  system — that  it  be  error  centered  and  that 
it  have  a  reference  mechanism  against  which  feedback  from  a  response 
can  be  compared  for  the  detection  and  correction  of  error.  In  this 
regard,  Bilodeau  and  Bilodeau  (1961)  point  out  that  the  central  focus 
of  an  open-loop  system  is  the  correct  response,  whereas  the  central 
focus  of  a  closed- loop  system  is  error. 

Until  recently,  all  learning  theories,  including  those  of 
James  (1890),  Washburn  (1916),  Lashley  (1917),  Thorndike  (1927),  and 
Bartlett  (1948)  were  open-looped  in  that  they  treated  errors 
incidentally  or  only  hinted  at  feedback  being  partically  responsible 
for  successive  responses.  Adams  (1971)  however,  has  proposed  a  theory 
of  motor  learning  with  error  as  its  central  focus.  In  Adams'  theory, 
the  reference  mechanism  or  "perceptual  trace"  against  which  the 
correctness  of  a  particular  response  is  evaluated  is  developed  and 
strengthened  as  a  function  of  sensory  feedback  received  from  the 
responses.  The  perceptual  trace  is  actually  a  motor  image  of  past 
movements  which  the  learner  uses  to  compare  against  the  feedback 
from  each  response.  Differences  between  the  perceptual  trace  and 
feedback  from  individual  responses  aid  the  learner  in  improving 
responses  until  discrepancies  between  the  feedback  and  the  perceptual 
trace  eventually  cease  to  exist.  At  this  stage  of  learning,  Adams 
maintains  that  the  learner  can  now  respond  with  respect  to  the  per¬ 
ceptual  trace  alone  and  feedback  concerning  responses  is  no  longer 
essential  to  the  maintenance  of  adequate  performance.  Although  he 
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contends  that  feedback  is  no  longer  essential  at  this  point  in 
training,  he  also  adds  that  the  continuance  of  feedback  in  the  transfer 
situation  will  enhance  performance. 

Even  though  no  general  agreement  exists  among  motor-learning 
theorists  as  to  the  precise  role  of  feedback  in  motor  skill 
acquisition,  they  do  agree  that  feedback  concerning  performance  level 
is  critical  in  the  acquisition  of  a  motor  skill.  Despite  this  fact, 
some  training  systems  inherently  eliminate  much  of  the  intrinsic  feed¬ 
back  available  to  the  trainee.  The  most  popular  of  such  systems  are 
those  that  employ  an  adaptive  logic. 

Adaptive  training  systems  are  of  interest  to  motor  skills 
researchers  since  they  provide  a  means  by  which  motor  skills  training 
may  be  individualized.  In  open-loop  training  systems  where  task 
difficulty  is  maintained  at  a  constant  level  throughout  training, 
the  task  is  often  too  easy  at  certain  points  in  the  training  process 
for  some  trainees  while  being  too  difficult  for  others.  In  an  adaptive 
training  system,  on  the  other  hand,  one  or  more  variables  important  to 
the  training  task  are  changed  according  to  the  trainee's  performance 
(see  Figure  2).  As  the  trainee's  performance  improves,  the  difficulty 
or  complexity  of  the  adaptive  variable(s)  is  manipulated  such  that  some 
measure  of  performance  remains  relatively  constant  throughout  training. 
As  a  result  of  this  adaptiveness  of  the  training  system,  the  learner's 
error  remains  relatively  constant  throughout  training.  Irrefutably, 
this  is  a  closed-loop  system  in  that  its  central  focus  is  error  and 
the  trainee's  performance  affects  the  system's  dynamics.  However, 
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error  in  a  closed-loop  system  is  to  be  used  by  the  trainee  in  evalu¬ 
ating  performance-  In  adaptive  training  systems,  the  error  perceived 
by  the  trainee  remains  relatively  constant,  leaving  little  on  which 
performance  may  be  judged  or  future  responses  based.  Kelley  (1969), 
an  innovator  of  adaptive  training,  recognized  this  fact  and  suggested 
(but  did  not  experimentally  test)  that  augmented  feedback  in  the  form 
of  a  meter  indicating  task  difficulty  level  and  possibly  an  indicator 
of  out-of-tolerance  performance  be  provided  to  trainees  in  adaptive 
training  systems. 

Augmented  feedback  in  motor  skills  training.  An  extensive  amount 
of  research  examining  the  effects  of  augmented  feedback  in  traditional, 
open-loop  motor  skills  training  systems  has  been  conducted,  but  the 
results  have  often  been  in  disagreement  with  one  another.  Archer, 

Kent,  and  Mote  (1956),  Archer  and  Namikas  (1958),  and  Bilodeau  and 
Rosenquist  (1964)  found  augmented  feedback  to  have  no  effect  on 
subjects'  performance  in  training  or  transfer.  Numerous  other  studies 
have  found  augmented  feedback  to  aid  performance  in  training  but  to 
have  no  effect  on  transfer  performance  when  feedback  is  withdrawn 
(Morin  and  Gagne,  1951;  Bilodeau,  1952;  Goldstein  and  Rittenhouse, 

1954;  Bilodeau,  1955;  Payne  and  Hauty,  1955;  Karlin  and  Mortimer,  1963; 
and  Sheldon  and  Bjorklund,  1966).  Other  researchers  have  found 
augmented  feedback  to  produce  a  learning  effect  where  the  performance 
of  the  feedback  groups  remains  significantly  better  than  that  of  the 
no-feedback  groups  even  after  the  augmented  feedback  is  withdrawn 
(Reynolds  and  Adams,  1953;  Smode,  1958;  Williams  and  Briggs,  1962; 


Kinkade,  1963;  Karlin,  1965;  Gordon  and  Gottlieb,  1967;  and 
Gordon,  1.968). 

Although  these  studies  have  produced  conflicting  results,  they 
have  been  valuable  in  formulating  some  general  conclusions  regarding 
the  use  of  augmented  feedback  in  open-loop  motor  skills  training 
systems.  Welford  (1968)  observed  that  supplemental  feedback  must  not 
provide  cues  which  are  eventually  relied  upon  to  the  extent  that  when 
the  feedback  is  withdrawn,  the  trainee  is  no  longer  able  to  maintain 
adequate  performance.  A  performance  decrement  upon  feedback  withdrawal 
has  most  often  been  found  in  systems  where  the  trainee  is  capable  of 
performing  part  of  the  task  by  simply  attending  to  the  supplemental 
feedback  Cues.  When  the  supplemental  feedback  is  withdrawn,  a 
considerable  amount  of  information  loss  occurs  and  performance 
deteriorates.  Annett  and  Kay  (1957)  suggest  that  if  such  cues  are 
to  be  used,  the  training  situation  should  be  arranged  so  that  the 
extra  cues,  while  aiding  the  trainee  in  the  early  part  of  training, 
later  become  redundant  information.  In  this  way,  the  trainee  would  no 
longer  rely  upon  these  cues  in  the  later  parts  of  training  and, 
therefore,  no  serious  decline  in  performance  would  occur  upon  their 
withdrawal.  If  one  puts  Annett  and  Kay's  suggestion  into  perspective 
with  Adams'  (1971)  theory  of  motor  learning,  it  is  apparent  that 
Annett  and  Kay  are  suggesting  the  removal  of  these  cues  when  the 
perceptual  trace  has  been  developed  to  the  point  that  feedback  is  no 
longer  essential  to  adequate  performance. 
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The  reliance  upon  supplemental  feedback  is  most  often  encountered 
in  predictable  systems  providing  rather  precise  error  information. 
Since  error  is  relatively  constant  in  adaptive  training  systems  and 
augmented  feedback  would  be  more  performance  oriented,  any  such 
problems  should  be  circumvented. 

Clarity  of  the  intrinsic  feedback  provided  by  the  training  task 
has  also  been  found  to  be  an  important  variable  interacting  with 
augmented  feedback  (Adams,  1964).  If  the  intrinsic  feedback  in  the 
training  system  is  not  clearly  discernible,  the  additional  feedback 
provided  will  become  a  part  of  the  total  information  used  to  guide 
performance.  Thus,  the  function  of  the  augmented  feedback  will  change 
from  its  intended  purpose  of  aiding  in  the  evaluation  of  performance 
to  one  of  guiding  performance.  Kinkade  (1963)  found  that  subjects 
who  received  augmented  feedback  in  a  training  task  where  the  intrinsic 
feedback  was  ambiguous  performed  significantly  better  in  training  than 
subjects  who  did  not  receive  augmented  feedback.  However,  when  feed¬ 
back  was  withdrawn  in  transfer,  the  performance  of  the  feedback  group 
declined  to  that  of  the  no-feedback  group.  In  a  similar  training  task 
in  which  the  intrinsic  feedback  was  clearly  discernible,  subjects 
provided  with  augmented  feedback  in  training  performed  significantly 
better  in  training  and  transfer  than  subjects  who  did  not  receive 
augmented  feedback  in  training.  These  findings  are  of  importance  in 
considering  the  implementation  of  augmented  feedback  into  any  training 


system. 
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Williams  and  Briggs  (1962)  concluded  Chat  Che  type  of  behavior 
indicated  by  augmented  feedback  is  related  to  the  effect  it  produces. 
With  feedback  indicating  out-of-tolerance  performance,  large  errors 
are  emphasized  early  in  training  and  the  trainee  learns  to  minimize 
them  quickly.  With  feedback  indicating  in-tolerance  performance, 
small  errors  are  emphasized  and  the  trainee  is  not  as  quick  in 
responding  to  correct  the  large  errors  not  emphasized  by  the  feed¬ 
back.  Since  minimizing  out-of-tolerance  performance  is  of  primary 
importance  in  training,  out-of-tolerance  feedback  would  appear  best 
in  aiding  the  trainee  to  achieve  the  desired  performance  level. 

Williams  and  Briggs  (1962)  also  made  the  important  observation 
that  with  feedback  indicating  out-of- tolerance  performance,  the  amount 
of  feedback  received  by  the  trainee  diminishes  as  training  continues. 
Thus,  when  the  trainee  is  put  into  the  transfer  situation  where  feed¬ 
back  is  not  present,  the  change  is  not  as  abrupt.  In  their  research, 
Williams  and  Briggs  found  that  subjects  who  received  out-of-tolerance 
feedback  in  training  did  not  exhibit  a  performance  decrement  in  the 
no-feedback  transfer  task.  Subjects  receiving  in-tolerance  feedback 
during  training,  however,  failed  to  maintain  their  superiority  over 
the  control  group  that  did  not  receive  feedback  in  training.  In 
addition,  the  in-tolerance  feedback  group  displayed  considerably 
greater  inter-subject  variability  in  transfer  than  the  out-of-tolerance 
feedback  group.  These  results  are  in  agreement  with  the  findings  of 
Adams,  Goetz,  and  Marshall  (1972),  which  suggest  that  the  performance 
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decrement  in  going  from  the  training  to  the  transfer  situation  is 
least  when  the  change  in  feedback  is  minimal. 

Finally,  Bilodeau  (1966)  and  Annett  (1969)  suggest  that  augmented 
feedback  should  direct  the  attention  of  the  trainees  to  the  results 
of  their  responses  and,  thus,  their  errors.  Out-of-tolerance  feedback 
would  appear  to  be  best  suited  for  this  in  that  the  trainees  would  be 
signaled  when  their  responses  are  in  error.  In  an  adaptive  training 
system,  an  indication  of  task  difficulty  would  provide  the  trainees 
with  similar  performance  information  in  that  a  decrease  in  task 
difficulty  would  Indicate  a  performance  decrement  while  an  increase 
in  task  difficulty  would  indicate  an  Improvement  in  performance. 

In  summary,  research  on  traditional,  open-loop  motor  skills 
training  suggests  that  in  order  for  augmented  feedback  to  have  a 
learning  effect  (i.e.,  to  have  the  improved  performance  caused  by 
augmented  feedback  to  continue  after  the  feedback  is  withdrawn) ,  the 
following  provisions  must  be  made:  (1)  the  augmented  feedback  must 
not  provide  cues  which  are  relied  upon  for  successful  performance 
throughout  training,  (2)  the  intrinsic  feedback  provided  by  the 
training  system  must  be  clearly  discernible,  (3)  the  augmented  feed¬ 
back  should  signal  out-of-tolerance  performance,  and  (4)  the  augmented 
feedback  should  direct  the  attention  of  the  trainees  to  the  results 
of  their  responses  and,  thus,  their  errors.  Although  these  results 
are  the  culmination  of  several  years  of  research  effort  on  traditional 
open- loop  motor  skills  training  systems,  they  can  not  be  generalized  to 
closed- loop  forms  of  motor  skills  training  without  empirical  testing. 
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Statement  of  the  Problem 

Research  conducted  on  the  effects  of  augmented  feedback  In 
closed- loop  motor  skills  training  systems  has  been  scant.  Norman 
(1973)  is  one  of  the  few  to  have  carried  out  such  research.  In  a 
series  of  experiments,  he  provided  subjects  in  an  adaptive  flight 
training  task  with  visually  augmented,  level-of-dlfficulty  feedback 
and  found  the  visual  feedback  to  have  no  effect  on  training  performance. 
However,  when  the  feedback  was  withdrawn  in  transfer,  subjects  who 
received  feedback  in  training  performed  significantly  better  than 
those  who  were  in  the  no-feedback  training  groups.  Unfortunately, 
the  reliability  of  these  findings  is  somewhat  questionable  since  the 
comparisons  were  made  between  groups  in  more  than  one  experiment  in 
which  different  subject  pools  were  used,  the  independent  variables 
differed,  and  the  training  task  was  substantially  different  from  the 
transfer  task. 

Cote,  Williges,  and  Williges  (1978)  conducted  two  studies  on 
the  use  of  augmented  feedback  in  motor  skill  training  using  a  two- 
dimensional  pursuit  tracking  task.  The  purpose  of  this  research  was 
to  evaluate  the  effects  of  visually  presented  augmented  feedback  on 
motor  skills  learning  using  either  traditional,  open-loop  training  or 
closed- loop  training.  Feedback  in  both  studies  was  in  terms  of  task 
difficulty  and  performance  accuracy  (see  Figure  3).  In  Study  I,  the 
effects  of  augmented  feedback  cues  on  adaptive  motor  learning  were 
examined  using  an  automatic  adaptive  training  procedure  in  which  task 
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difficulty  increased  as  a  function  of  the  trainee's  performance.  Six 
male  subjects  were  randomly  assigned  to  each  of  the  four  treatment 
conditions  presented  in  Table  1. 

Although  the  mean  training  time-to-criterlon  scores  of  subjects 
who  received  augmented  feedback  was  not  significantly  higher  than 
that  of  subjects  who  did  not  receive  the  augmented  feedback,  there  was 
a  trend  for  the  training  scores  of  subjects  who  received  feedback  to 
be  higher.  A  possible  explanation  for  this  may  come  from  the  comments 
of  several  subjects  who  mentioned  that  when  they  attempted  to  use  the 
augmented  feedback,  their  tracking  performance  declined  sharply.  In 
transfer,  no  reliable  effects  due  to  feedback  in  training  and/or 
transfer  were  observed  (g^  >  0.10). 

In  Study  II,  the  combined  effects  of  visually  presented  augmented 
feedback  and  training  procedure  were  examined.  Two  training  pro¬ 
cedures  were  used:  (1)  a  fixed-difficulty  procedure  in  which  the 
trainee  was  presented  with  the  criterion  level  of  task  difficulty 
throughout  training,  and  (2)  the  automatic  adaptive  training  procedure 
used  in  Study  I. 

Six  male  subjects  were  randomly  assigned  to  each  of  four  training/ 
feedback  conditions.  These  were:  (1)  a  fixed-difficulty  procedure 
in  which  no  augmented  feedback  was  presented,  (2)  a  fixed-difficulty 
procedure  in  which  visual  augmented  feedback  was  presented,  (3)  an 
automatic  adaptive  procedure  in  which  no  augmented  feedback  was 
presented,  and  (4)  an  automatic  adaptive  procedure  in  which  visual 
augmented  feedback  was  presented.  A  transfer  task  similar  to  the 


Condition 

Training 

Transfer 

I 

Feedback 

Feedback 

II 

Feedback 

No  Feedback 

III 

No  Feedback 

Feedback 

IV 


No  Feedback 


No  Feedback 
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training  task  was  given  to  all  groups  in  which  feedback  was  not 
presented.  Results  of  the  analysis  of  variance  indicated  no  differ¬ 
ence  in  training  time-to-criterion  (£  >  0.05).  Thus,  visually 
presented  augmented  feedback  did  not  aid  subjects  in  either  type  of 
training  procedure.  Furthermore,  subjects  trained  adaptively  required 
the  same  amount  of  training  as  those  in  the  fixed-difficulty  conditions. 
However,  subjects  who  were  trained  adaptively  performed  reliably 
better  in  transfer  than  those  receiving  fixed-difficulty  training 
(£  *  0.0065).  The  presence  of  feedback  in  training  did  not  prove  to 
have  any  effect  on  transfer  performance  (£  >  0.10). 

The  results  of  the  above  two  studies  imply  that  constantly 
presented  visual  feedback  produces  neither  a  performance  nor  a 
learning  effect  in  a  complex,  closed-loop  tracking  task.  However, 
based  on  the  conclusions  from  open-loop  motor  skills  research,  it  would 
appear  that  augmented  feedback  should  have  enhanced  performance 
in  training  and  transfer.  From  the  training  results  of  Study  I  and 
the  comments  of  several  subjects  in  the  feedback  conditions,  there 
was  reason  to  believe  that  the  training  task  in  these  two  studies 
Imposed  such  a  large  visual  workload  upon  the  subjects  that  they  were 
unable  to  use  the  feedback  effectively.  In  fact,  the  visual  workload 
of  the  subjects  in  the  feedback  conditions  was  theoretically  Increased 
by  providing  them  with  the  visual  feedback. 

If  the  workload  imposed  by  a  task  is  such  that  the  augmented 
feedback  can  not  be  attended  to,  then  the  feedback  will  not  have  an 
effect  on  training  performance.  With  this  being  the  case,  one  can  not 
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expect  transfer  performance  In  a  non-augmented  feedback  condition 
to  be  affected  either.  This  has  been  a  consistent  finding  of 
augmented  feedback  research  in  which  augmented  feedback  failed  to 
enhance  behavior  while  present  in  training. 

Thus,  a  primary  concern  when  employing  augmented  feedback  should 
be  its  capability  of  being  used  by  the  trainee.  In  relatively  simple 
systems,  this  criterion  may  be  met  by  presenting  augmented  feedback 
in  any  one  of  numerous  ways.  However,  the  more  complex  the  training 
system,  the  more  careful  one  must  be  in  presenting  augmented  feed¬ 
back  so  that  it  is  useful.  For  example,  in  complex  training  systems 
outside  of  the  laboratory,  a  high  visual  workload  is  often  encountered. 
Some  systems  require  the  operator  to  constantly  monitor  numerous 
continuous  Information  displays  which  provide  information  important 
for  successful  task  performance.  One  such  instance  is  a  flight 
training  simulator.  In  situations  such  as  this,  it  is  imperative 
that  means  other  than  the  constant  presentation  of  visual  feedback 
be  explored  if,  indeed,  this  augmented  information  is  capable  of 
aiding  performance. 

Some  research  in  this  area  has  recently  been  conducted.  Gibson 
and  Ventola  (1976)  successfully  employed  constantly  augmenting  tactile 
feedback  to  present  pilots  with  flight  path  information  during 
approach  and  landing  operations.  Lintem  (1978)  presented  adaptive 
visual  flight  path  information  during  landing  operations  that  appeared 
only  when  the  trainee  was  off-course.  As  in  Gibson  and  Ventola's  study, 


Lintern  found  a  performance  as  well  as  a  learning  effect  attributable 
to  the  augmented  feedback. 

Purpose  of  Thesis  Research 

Research  dealing  with  the  effects  of  augmented  feedback  on  closed- 
loop  motor  skills  training  has  been  severely  limited.  Furthermore, 
little  has  been  done  to  investigate  alternative  procedures  for  pre¬ 
senting  augmented  feedback  in  complex  motor  skills  training,  the 
purpose  of  the  present  research  was  to  evaluate  the  effects  of  two 
types  of  off-course  augmented  feedback  using  two  training  procedures 
on  the  learning  of  a  complex  motor  skill. 

The  main  point  of  interest  in  this  study  was  the  effect  of  the 
augmented  feedback  on  adaptive  motor  skill  learning.  The  types  of 
-feedback  used  were:  (1)  auditory  feedback,  (2)  visual  feedback, 

(3)  auditory  plus  visual  feedback,  and  (4)  no  feedback.  Two  training 
procedures  were  used  to  assess  whether  or  not  the  various  augmented 
feedback  types  would  have  any  differential  effects  upon  training 
procedure.  The  training  procedures  used  were:  (1)  a  fixed-diffi¬ 
culty  training  procedure  representing  traditional,  open-loop  motor 
skills  training,  and  (2)  an  adaptive  training  procedure. 

With  the  use  of  off-course  augmented  feedback,  the  amount  of 
information  presented  diminished  as  training  progressed.  This 
prevented  an  abrupt  change  in  the  amount  of  information  available 
from  occuring  in  the  transition  from  training  to  the  no-feedback 
transfer  condition.  Furthermore,  off-course  feedback  called  attention 


Co  and  emphasized  one's  errors  early  in  training.  In  addition  to 
these  desirable  qualities,  off-course  information,  if  correctly 
implemented,  has  a  clear  advantage  over  continuous  augmented  infor¬ 
mation  in  that  it  does  not  require  constant  monitoring. 

Besides  off-course  Information,  subjects  who  received  feedback, 
in  the  adaptive  training  procedure  also  were  provided  information 
concerning  the  changing  levels  of  task  difficulty.  The  decision 
to  include  such  information  was  the  result  of  a  suggestion  by  Kelley 
(1969).  With  error  remaining  relatively  constant  in  an  adaptive 
training  system,  Kelley  suggested  that  it  is  important  for  the  trainee 
to  be  provided  with  some  type  of  information  indicating  performance 
Improvement . 

It  was  hypothesized  that  the  augmented  feedback  would  produce 

* 

a  performance  as  well  as  a  learning  effect  in  both  types  of  training 
procedures  used.  It  was  also  hypothesized  that  subjects  who  received 
adaptive  training  and  augmented  feedback  would  have  lower  training 
scores  than  subjects  in  the  fixed  difficulty  conditions  receiving 
augmented  feedback.  Providing  level-of-dif ficulty  feedback  with 
adaptive  training  eliminates  the  problem  of  there  being  no  performance 
information  in  adaptive  systems.  Thus,  the  combination  of  the  adaptive 
procedure  which  provides  tailored  training  and  level  of  difficulty 
feedback  which  provides  performance  information,  training  should  be 
facilitated  in  terms  of  lower  time-to-criterion  scores.  Finally,  it 
was  hypothesized  that  those  who  received  adaptive  training  would 
perform  significantly  better  in  the  transfer  task  with  multiple  levels 
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of  Cask  difficulty  Chan  Chose  who  received  fixed-difficulty  training. 
This  hypothesis  was  based  on  the  fact  that  subjects  presented  witn  the 
adaptive  training  procedure  have  more  experience  in  tracking  the 
forcing  function  at  various  levels  of  difficulty  and  therefore  could 
more  readily  adapt  to  changing  levels  of  task  difficulty  in  transfer. 


METHOD 


Tracking  Task 

To  investigate  the  possible  facilitating  effect  of  augmented 
feedback  in  motor  skills  training,  a  two-dimensional  pursuit  tracking 
task  was  used.  The  experiment  was  divided  into  two  sessions,  a 
training  session  and  a  transfer  session,  with  the  same  tracking  task 
being  used  in  both  sessions. 

Training  task.  Subjects  learned  the  two-dimensional  pursuit 
tracking  task  illustrated  in  Figure  4.  Three  independent,  random, 
band-limited  functions  were  used  to  determine  the  forcing  function 
of  the  pursuit  symbol  ("X")  on  the  display.  The  band-limited  functions 
determined  the  length  of  a  movement  in  each  axis  and  the  duration  of 
the  vector  movement.  The  forcing  function  of  the  tracking  symbol 
("0")  was  generated  from  the  output  of  an  isometric  control  stick. 

The  effective  tracking  area  on  the  display  was  12.7  cm  X  12.7  cm 
with  the  visual  feedback  appearing  outside  of  this  region  (8.89  cm 
from  the  display's  center).  Subjects  rested  their  heads  on  a  fore¬ 
head  rest  such  that  the  viewing  distance  was  kept  constant  at  1  m. 

With  this  viewing  distance  and  a  subject's  point  of  regard  being  the 
center  of  the  display,  the  visual  angle  subtended  by  the  feedback  bars 
was  10.1°.  Thus,  the  feedback  bars  were  in  peripheral  vision  when  a 
subject  fixated  on  the  center  of  the  display.  Each  tracking  symbol 
presented  on  the  display  occupied  a  0.64  cm  X  0.48  cm  area. 
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The  control  system  dynamics  of  this  system  were  neither  pure 
rate  nor  pure  acceleration.  With  pure  acceleration  system  dynamics, 
pretesting  results  showed  mean  time  to  be  more  than  1  h,  a  time 
period  longer  than  that  desired  by  the  experimenter.  In  pretesting 
subjects  with  a  pure  rate  system,  mean  training  time  was  less  than 
5  min  with  little  intersubject  variability.  Thus,  various  weighting 
constants  were  placed  on  both  control  orders  and  several  new  subjects 
were  pretested  to  establish  weightings  that  would  result  in  training 
times  acceptable  to  the  experimenter  and  also  yield  satisfactory 
intersubject  variability. 

For  each  stick  output,  the  corresponding  positions  of  the 
"0"  under  both  pure  rate  and  pure  acceleration  control  were  calcu¬ 
lated.  The  following  transfer  function  describes  the  system  dynamics 
in  each  axis: 


0 
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where  0qx  (S)  *  the  Laplace  transform  of  the  scalor  change 
in  position  of  the  "0"  in  one  axis 
*  -  a  weighting  constant  used  to  weight  rate  control 
and  acceleration  control  on  a  relative  basis 


*  gain  constant 
Kj  ■  time  constant 

■  input  force  on  the  control  stick  in  one  axis 
S  *’  the  Laplace  transform  independent  variable. 

For  this  experiment,  *  was  specified  equal  to  0.8.  The  constants 
and  Kj  were  derived  as  a  function  of  the  maximum  allowable  speed 
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of  Che  "0"  set  in  Che  Cask  parameters  (see  Appendix  A)  and  the 
effective  Cracking  area  on  the  display.  These  constants  were 
specified  as  follows: 

Kx  -  1.4511  cm  s"1  N  1 
K2  *  1,6  s*1 

The  input  force  was  that  applied  to  the  top  of  the  12.54  cm  isometric 
controller  and  measured  at  its  base. 

The  random  movement  of  the  pursuit  symbox  was  generated  by 
simulating  a  computer-operated  control  stick  identical  to  the  transfer 
function  of  the  isometric  controller  as  described  in  Equation  1.  In 
Equation  2,  would  be  the  simulated  control  stick  output  value.  The 
K.  constants  for  the  pursuit  symbol  were  derived  as  a  function  of  the 
maximum  and  minimum  velocities  of  the  "X"  set  in  the  parameters  of 
the  task  and  the  effective  tracking  area  on  the  display.  These 
constants  were  specified  as  follows: 

K  -  0.7246  cm  s'1  iT1 
K2  -  0.8  s"1 

The  value  for  the  input  force  in  newtons  was  chosen  from  a  random 
number  generator  internal  to  the  mini-computer  used  to  generate  the 
experimental  task.  Task  difficulty  was  defined  in  terms  of  the 
movement  speed  of  the  pursuit  symbol  and  changed  relative  to  the  output 


value  of  the  simulated  control  stick. 
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A  small-step  adaptive  logic  was  available  to  vary  task  difficulty 
by  changing  the  maximum  allowable  output  value  of  the  simulated  control 
stick.  Absolute  vector  tracking  error  was  computed  every  60  ms  and 
compared  to  a  tolerance  limit  of  10%  of  the  screen  diagonal  (1.8  cm). 

A  total  of  1851  task  difficulty  steps  requiring  a  minimum  time  of 
111.1  s  to  reach  the  exit  criterion  level  of  difficulty  was  used. 

With  such  a  small-step  adaptive  logic,  performance  was  stabilized 
throughout  training,  but  increases  in  level  of  task  difficulty  were 
not  readily  apparent  from  the  task  Itself. 

The  criterion  level  of  task  difficulty  during  training  was  a 
maximum  possible  movement  speed  of  the  "X"  of  20.32  cm/s.  In  the 
fixed-difficulty  training  procedure,  the  speed  of  the  "X"  was  main¬ 
tained  at  the  criterion  level  throughout  training.  Exit  criterion 
training  performance  was  the  maintenance  of  the  "0"  within  10%  of 
the  effective  screen  diagonal  of  the  "X"  for  a  period  of  20  continuous 
seconds  while  the  "X"  was  moving  at  the  criterion  level  of  difficulty. 
To  avoid  confounding  fatigue  effects,  subjects  were  given  a  maximum 
of  fifteen,  3-min  trials  interspersed  with  1-min  rest  periods  to  exit 
the  training  task. 

Transfer  task.  After  training  to  criterion  and  a  5-min  rest 
period,  subjects  were  given  a  6-rnin  transfer  task  Identical  to  the 
training  task  with  the  exception  that  no  feedback  was  presented  to 
any  of  the  groups.  Task  difficulty  shifted  among  three  levels  with 
each  level  of  difficulty  being  presented  for  2  min.  These  three  levels 
of  difficulty  were:  the  exit  criterion  level  of  difficulty  in  training 
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(20.32  cm/s),  0.5  X  exit  criterion  level  (10.16  cm/s),  and  1.5  X 
exit  criterion  level  (30.48  cm/s).  The  presentation  of  the  levels 
of  difficulty  was  completely  balanced  within  each  cell,  yielding  six 
possible  presentation  orders.  With  there  being  12  subjects  per  cell, 
two  subjects  in  each  cell  received  the  same  presentation  order. 

Experimental  Design. 

Figure  5  contains  a  block  diagram  of  the  experimental  design. 

The  two  factors  examined  in  training  were  training  procedure  and 
augmented  feedback.  Training  procedure  had  two  levels  with  one 
being  adaptive  training  (closed-loop)  in  which  the  level  of  task 
difficulty  varied  as  a  function  of  the  trainee's  performance  according 
to  the  logic  described  in  the  training  task  and  the  other  being 
fixed-difficulty  training  (open-loop)  in  which  the  trainee  was  pre¬ 
sented  with  the  criterion  level  of  task  difficulty  throughout  training. 
The  second  factor  examined  in  training,  augmented  feedback,  had  four 
levels.  These  four  levels  included  two  different  types  of  off-course 
feedback  (visual  and  auditory),  a  combination  of  both  types  of  off- 
course  feedback,  and  no  augmented  feedback. 

Visual  feedback  was  provided  as  shown  in  Figure  4.  On  the  left 
hand  side  of  the  display  appeared  a  bar  graph  labeled  "ACCURACY." 

The  horizontal  line  at  the  top  of  the  bar  graph  indicated  in-tolerance 
performance,  or,  the  "0"  being  within  10%  of  the  effective  screen 
diagonal  of  the  "X"  (1.8  cm).  When  the  "0"  was  closer  than  1.8  cm  to 
the  "X",  the  vertical  line  disappeared  from  the  display.  If  the 
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Figure  5.  Block  diagram  of  experimental  design. 
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subject  moved  the  "0"  outside  of  the  tolerance  range,  the  bar  re¬ 
appeared  on  the  display.  On  the  right  hand  side  of  the  display 
appeared  a  bar  graph  labeled  "DIFFICULTY."  As  task  difficulty  in¬ 
creased,  the  bar  approached  the  upper  horizontal  line  which  indicated 
criterion  task  difficulty.  When  criterion  task  difficulty  was 
achieved,  the  right  hand  bar  graph  disappeared  from  the  display.  In 
the  adaptive  training  procedure,  the  bar  graph  reappeared  if  a 
subject's  performance  became  out-of-tolerance  after  the  criterion 
level  of  difficulty  had  been  reached.  In  the  fixed-difficulty 
training  procedure,  this  bar  graph  was  not  presented  because  task 
difficulty  was  always  at  the  criterion  level. 

Auditory  feedback  was  provided  through  a  loudspeaker  located 
approximately  1.5  m  to  the  rear  and  1.5  m  above  the  subjects.  Task 
difficulty  was  indicated  by  50  ms  "beeps"  at  400  Hz  and  46  dB.  When 
the  task  was  at  its  lowest  level  of  difficulty  (as  it  was  when  training 
began) ,  the  beeps  were  presented  at  the  rate  of  20/s  and  were  perceived 
as  a  continuous  tone.  As  task  difficulty  increased,  the  period  be¬ 
tween  beeps  also  increased  until  the  criterion  level  of  task  difficulty 
was  reached.  At  this  point  the  beeps  were  no  longer  presented.  How¬ 
ever,  if  performance  decrement  occurred  and  task  difficulty  decreased, 
the  beeps  reappeared  at  an  initial  rate  of  one  beep  every  400  ms.  If 
performance  decrement  continued  to  occur,  the  rate  of  the  beeps  con¬ 
tinued  to  increase. 

Tracking  accuracy  was  indicated  by  a  continuous,  2,000  Hz  tone 
varying  in  amplitude  from  46  dB  to  51  dB.  When  the  "0"  was  more  than 


30 


10%  of  Che  effecCive  screen  diagonal  away  from  Che  "X"  for  a  period 
of  60  ms,  onseC  of  Che  Cone  occurred  ac  46  dB  wlch  Che  cone  Increasing 
in  amplicude  Co  51  dB  when  Che  subjecC  was  >_  50%  of  Che  effecCive 
screen  diagonal  away  from  Che  "X1'.  All  frequencies  and  amplicudes  were 
selecced  such  ChaC  no  masking  occurred. 

As  can  be  seen,  Che  off-course  feedback  employed  in  this  sCudy 
was  such  chac  ic  was  nocicable  when  presenced  and  therefore  did  noC 
require  Che  subjecC  Co  perform  a  secondary  Cask  of  moni Coring  for 
ics  presence.  FurChermore,  although  the  on-target  versus  off-target 
augmented  feedback  may  have  been  redundant  information  in  chat  this 
information  was  readily  available  from  the  task  itself,  the  task 
difficulty  augmented  feedback  given  in  the  adaptive  training  procedure 
provided  information  not  readily  apparent  from  the  task  due  to  the 
small-step  adaptive  logic  employed. 

Equipment 

The  cracking  task  was  generated  using  a  laboratory  developed 
software  package  run  on  a  Digital  Equipment  Corporation  PDP  11/55 
digital  computer.  The  task  was  displayed  on  a  Tektronix  4014-1 
computer  display  terminal.  Subjects  tracked  the  pursuit  symbol  using 
a  Measurement  Systems  Model  435  two-axis  isometric  control  stick. 
Augmented  feedback  was  presented  visually  on  the  Tektronix  display  and 
auditorily  through  a  Utah  15.24  cm,  general  purpose  loudspeaker.  The 
hardware  responsible  for  generating  the  auditory  signals  was  built 


in-house. 
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For  the  2,000  Hz  continuous  tone  indicating  in-tolerance  versus 
out-of-tolerance  performance,  an  analog  voltage  was  sent  from  the 
D/A  converter  of  the  PDP  11/55  (a  DEC  LPS  11)  which  was  proportional 
to  the  subject's  distance  (in  terms  of  percent  of  screen  diagonal) 
from  the  pursuit  symbol.  This  voltage  was  received  by  a  voltage 
control  amplifier  which  also  had  a  2,000  Hz  input.  If  the  voltage 
received  corresponded  to  a  subject's  tracking  error  being  greater  than 
10%  of  the  screen  diagonal  (out-of-tolerance  performance) ,  the  voltage 
control  amplifier  amplified  the  2,000  Hz  input  to  a  specific  dB  level 
between  46  dB  and  51  dB  and  sent  it  to  another  amplifier  which 
powered  the  Utah  speaker.  If  subject  error  was  less  than  10%  of  the 
screen  diagonal,  no  auditory  signal  was  generated. 

The  beep  rate  of  the  400  Hz  tone  was  generated  as  the  result  of 
a  digital  output  of  the  LPS  11.  A  16  bit  word  with  a  numeric  value 
corresponding  to  task  difficulty  level  was  received  by  a  counter 
with  a  continuous  clock  input.  Given  che  digital  signal  and  the  input 
from  the  continuous  clock,  the  counter  determined  when  to  send  a 
signal  to  a  trigger  which  opened  a  circuit  to  a  400  Hz  input.  (Upon 
signaling  the  trigger,  the  counter  was  reset).  A  400  Hz  output  with 
a  specific  amplitude  and  duration  was  then  sent  to  the  same  amplifier 
which  powered  the  Utah  speaker  for  the  2,000  Hz  tone.  So  that  both 
auditory  signals  could  be  put  through  the  same  speaker,  an  analog 
adder  was  employed.  Thus,  before  either  auditory  signal  was  fed  into 
the  speaker  amplifier,  they  were  sent  through  the  analog  adder. 
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Figure  6  illustrates  the  experimental  set-up.  The  software 
package  responsible  for  generating  the  tracking  task  and  the  augmented 
feedback  used  two  independent  re«d-time  cycles— a  60-Hz  cycle  to 
refresh  the  display  and  a  60-ms  cycle  to  update  the  task  and  the 
augmented  feedback.  Figure  7  gives  an  overview  of  the  software 
package  while  Figure  8  contains  a  flow  diagram  of  the  real-time 
process.  Appendix  C  contains  a  listing  of  the  software  package. 

Subjects 

Twelve  paid  volunteer  male  college  students  were  randomly 
assigned  to  each  training/ feedback  condition  for  a  total  of  96 
subjects.  All  were  right-handed  and  naive  to  the  experimental  task. 
Before  participating  in  the  study,  all  subjects  were  given  a  full 
vision  test  with  a  Bausch  and  Lomb  Orthorater  and  were  required  to 
have  at  least  20/25  vision  (near  and  far,  corrected  or  uncorrected). 
Furthermore,  all  subjects  read  and  signed  a  consent  form  to  parti¬ 
cipate  in  the  study  which  delineated  their  rights  as  a  subject. 

Subjects  were  given  the  opportunity  to  receive  a  summary  of  the 
experimental  results  by  indicating  so  on  the  consent  form.  Appendix 
D  contains  the  summary.  Only  male  subjects  were  used  to  eliminate 
any  sex  differences  that  may  exist  in  motor  skills  training  (Williges, 
Williges,  and  Savage,  1977). 

Procedure 

Subjects  were  seated  in  the  experimental  room  and  given  sufficient 
time  to  adapt  to  the  room's  low  illuminance  level.  They  were  then 
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SUBJECT 


EXPERIMENTER 


Figure  6.  Experimental  set-up. 

(Prepared  by  John  Evans ,  III) . 


Figure  7.  Software  package. 

(Prepared  by  John  Evans,  III). 
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given  tape-recorded  instructions  appropriate  to  their  respective 
experimental  conditions.  After  attending  to  the  recording,  subjects 
were  given  the  opportunity  to  ask  questions  concerning  the  instructions. 
The  experimenter  remained  in  the  experimental  room  for  the  first 
trial  of  Che  training  session  after  which  the  subjects  were  once  again 
asked  if  they  had  any  questions.  For  the  remainder  of  the  experiment, 
Che  experimenter  resided  in  an  adjacent  room  to  which  the  subjects 
were  provided  communication  via  an  intercom  system. 


RESULTS 


Eight  subjects  failed  to  exit  from  the  training  task  in  the 

allotted  15  trials.  Three  were  in  the  fixed-difficulty  without 

augmented  feedback  condition  while  each  of  the  following  conditions 

had  one  no-exit  subject:  adaptive  training  with  auditory  feedback, 

adaptive  training  with  visual  feedback,  fixed  training  with  auditory 

feedback,  fixed  training  with  visual  feedback,  and  fixed  training 

with  auditory  and  visual  feedback.  Thus,  a  total  of  104  subjects 

were  required  to  balance  each  of  the  right  conditions  with  12 
2 

subjects.  Three  x  analyses  were  performed  with  the  no-exit  frequency 
data  to  determine  if  there  was  a  significant  difference  in  the  number 
of  no-exit  subjects  due  to  training  procedure,  type  of  augmented 
feedback  or  training/ feedback  conditions.  None  of  the  three  analyses 
yielded  significant  results  (£  >  0.10).  It  must  be  noted,  however, 
that  a  bias  in  favor  of  failing  to  reject  the  null  hypothesis  was 
introduced  in  those  analyses  in  that  some  of  the  expected  cell  fre¬ 
quencies  were  less  than  5  (Lewis  and  Burke,  1949).  As  a  result,  the 
probability  of  occurrence  of  the  alternative  hypothesis  (there  being 
a  significant  difference  in  the  number  of  no-exit  subjects  due  to 
training  procedure,  type  of  augmented  feedback  or  training/ f eedback 
conditions)  was  actually  higher  than  that  obtained  by  these  analyses. 

All  subjects  were  presented  the  transfer  task,  whether  or  not 
they  exited  from  training  within  the  allotted  15,  3-min  trials.  Equal 
n's  analyses  of  variance  as  well  as  unequal  n's  analyses  of  variance 
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were  performed  on  Che  data.  In  Che  unequal  n's  analysis  of  variance 
on  Che  training  data,  45  min  was  used  as  Che  time-to-criterion  for 
subjects  who  failed  Co  exit  from  Che  training  Cask.  The  unequal  n's 
ANOVA  used  was  Yate's  (1933)  weighted  squares  of  means  technique  in 
which  values  for  Che  missing  observations  in  each  cell  are  inserted 
so  as  Co  obtain  a  complete  set  of  data.  This  technique  yields 
inflated  treatment  F-values,  but  the  inflation  of  these  F-values 
is  modest  when  the  fraction  of  observations  inserted  in  each  cell  is 
small  relative  to  the  total  number  of  cell-observations  used  in  the 
analysis.  The  significant  findings  in  the  unequal  n's  analyses  were 
the  same  as  those  in  the  equal  n's  analyses.  (See  Appendix  G  for  the 
ANOVA  summary  tables  of  the  unequal  n's  analyses).  Thus,  the  values 
reported  are  those  of  the  equal  n's  analyses. 

Training 

Table  2  lists  the  mean  time-to-criterion  scores  and  their 
standard  deviations  for  the  eight  training/ feedback  groups.  A  two- 
way  analysis  of  variance  on  time-to-criterion  scores  yielded  no 
reliable  effect  due  to  training  procedure  or  augmented  feedback 
on  training  time  (£  >  0.25).  Table  3  presents  the  ANOVA  summary  table 
of  the  training  analysis. 

Transfer 

An  analysis  of  variance  on  vector  RMS  tracking  error  integrated 
over  each  minute  of  the  transfer  task  was  conducted  with  training 
procedure,  feedback  in  training,  and  level  of  task  difficulty  during 
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TABLE  2 


Mean  Time-to-Criterion  Scores  and  Standard  Deviations 


Condition 

Fixed  training/ Auditory  feedback 

14.3 

7.7 

Fixed  training/Visual  feedback 

13.4 

7.1 

Fixed  training/Auditory  plus  visual  feedback 

12.0 

8.7 

Fixed  training/No  feedback 

17.0 

9.8 

Adaptive  training/Auditory  feedback 

14.6 

9.3 

Adaptive  training/Visual  feedback  • 

14.2 

9.8 

Adaptive  training/Auditory  plus  visual  feedback 

15.3 

11.3 

Adaptive  training/No  feedback 

17.7 

6.8 

X,  Fixed  training:  14.2  min 
X,  Adaptive  training:  15.4  min 
X,  Feedback:  14.0  min 
X,  No  feedback:  17.4  min 
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TABLE  3 


ANOVA  Summary 

Table 

for  Equal  n  Training  Data 

Source 

df 

SS 

F 

£ 

Training  Procedure  (T) 

1 

146005.08 

0.51 

0.4776 

Feedback  Type  (F) 

3 

761279.64 

0.88 

0.4548 

T  x  F 

3 

119049.35 

0.14 

0.9332 

S/TF 

88 

25261347.63 

Total 

95 

26287681.70 

j 


transfer  as  factors.  Table  4  provides  a  summary  of  the  mean  error 
and  standard  deviation  for  the  right  training/f eedback  groups. 

The  main  effect  of  level  of  difficulty  was  significant  (£  <  0.0001) 
indicating  that  the  three  levels  of  task  difficulty  presented  in  trans¬ 
fer  did  represent  different  skill  levels  (see  Table  5).  Tracking 
error  increased  with  greater  task  difficulty  (6.4Z,  9 .4%  and  12. 3Z, 
respectively) . 

A  more  important  result  from  the  analysis  of  variance  was  the 
finding  that  training  procedure  had  a  significant  effect  (£  ■  0.0251) 
upon  transfer  performance.  The  mean  vector  error  of  those  trained 
adaptively  was  9.0Z,  whereas  the  mean  vector  error  on  those  trained 
in  the  fixed-difficulty  condition  was  9.7Z.  The  main  effect  of  feed¬ 
back  in  training  and  none  of  the  interactions  were  reliable. 

Questionnaire 

2 

A  x  analysis  was  performed  on  the  cumulative  responses  to  each 

2 

response  on  the  questionnaire.  A  summary  of  the  x  analysis  on  all 
questions  Is  provided  in  Appendix  F.  Several  of  these  findings  are 
of  primary  interest. 

Of  the  72  subjects  who  received  some  form  of  augmented  feedback, 

only  two,  a  significant  minority,  responded  that  they  did  not  employ 

2 

it  in  performing  the  task,  x  (D  ■  64.2,  £  <  0.005.  Out  of  the  70 

subjects  who  used  the  feedback,  a  significantly  greater  number  said 

they  found  the  feedback  to  be  more  than  "helpful"  on  the 

2 

questionnaire  they  were  administered,  x  (1)  ■  16.5,  £  <  0.025. 


However,  in  an  analysis  by  condition  on  the  helpfulness 
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TABLE  4 

Transfer  Vector  RMS  Tracking  Error  and  Standard  Deviations 

Mean  Error 

Conditions  (%  of  screen  diagonal)  s. 


Fixed/ Audi to  ry 

9.2 

2.6 

Fixed/Visual 

9.3 

2.8 

Fixed/Auditory  plus  Visual 

10.1 

3.5 

Fixed/No  Feedback 

10.4 

4.0 

Adaptive/Auditory 

9.1 

3.0 

Adaptive/Visual 

9.0 

3.1 

Adaptive/Auditory  plus  Visual 

9.3 

3.0 

Adaptive/No  Feedback 

8.6 

3.0 

X,  Fixed  Training:  9.7% 

X,  Adaptive  Training:  9.0% 
X,  Feedback:  9.3% 

X,  No  Feedback:  9.5% 
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TABLE  5 

ANOVA  Summary  Table  for  Equal  n  Transfer  Data 


Source 

df 

SS 

F 

JE. 

Between  Subject 

Training  Procedure  (T) 

1 

0.0041 

5.19 

0.0251 

Feedback  Type  (F) 

3 

0.0015 

0.65 

0.5870 

T  x  F 

3 

0.0030 

1.26 

0.2942 

S/TF 

88 

0.0698 

Within  Sublect 

Level  of  Difficulty  (LOD) 

2 

0.1667 

349 . 12 

0.0001 

LOD  x  T 

2 

0.0005 

1.16 

0.3155 

LOD  x  F 

6 

0.0005 

0.34 

0.9162 

LOD  x  T  x  F 

6 

0.0025 

1.75 

0.1114 

LOD  x  S/T 

176 

0.0420 

Total 

287 

0.2906 
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racing,  it  was  found  that  Che  condiClon  a  subject  was  in  significancly 

2 

affecCed  Che  helpfulness  racing,  x  (15)  ■  29.8,  £  <  0.025.  Approxi- 

raaCely  one-half  of  Chose  subjeecs  who  received  che  fixed  Craining 

wich  visual  feedback  and  Che  adapcive  Craining  wlch  boCh  types  of 

feedback  found  che  augmented  feedback  less  chan  "helpful." 

Another  interesting,  statistically  significant  finding  was 

subject  preference  to  feedback  type.  Significantly  more  subjects 

2 

indicated  that  they  preferred  auditory  to  visual  feedback,  x  (1)  ■ 

8.16,  £  <  0.005.  However,  this  preference  was  significantly  affected 

2 

by  training  procedure,  x  (1)  *  6.32,  £  <  0.005,  with  all  subjects  in 
the  fixed-difficulty  training  procedure  preferring  auditory  feedback 
while  only  7  subjects  (58. 3Z)  in  the  adaptive  training  procedure 
preferred  the  auditory  feedback. 

In  addition  to  the  above  significant  findings,  some  non-signifi¬ 
cant  findings  also  proved  to  be  interesting.  A  major  concern  of  many 
experimenters  in  employing  augmented  feedback  is  its  motivating 
effects.  However,  in  responding  to  a  question  asking  subjects  how 
interesting  they  found  the  experiment,  those  who  received  feedback 
found  the  experiment  no  more  Interesting  than  subjects  not  provided 
with  feedback  (£  >  0.10).  Another  Interesting  nonsignificant  finding 
was  that  the  actual  use  of  the  auditory  signal,  as  indicated  by  the 
subjects,  was  not  greater  than  that  of  the  visual  signal  <£  >  0.10) 
even  though  a  significantly  greater  number  of  subjects  preferred  the 
auditory  signal  over  the  visual  signal. 


DISCUSSION 


Augmented  Feedback 

Training.  In  training,  no  significant  effect  due  to  augmented 
feedback  was  found.  This  result  is  consistent  with  earlier  findings 
of  Cote,  e£  al.  (1978).  However,  the  results  of  this  study  do  not 
support  the  hypothesis  that  the  lack  of  an  augmented  feedback  effect 
with  this  task  is  due  to  a  visual  information  overload.  In  the  present 
study,  the  visual  attention  required  to  attend  effectively  to  the 
visual  feedback  was  minimal.  Furthermore,  the  auditory  feedback 
required  no  visual  attention. 

One  could  hypothesize  that  the  lack  of  a  feedback  effect  could 
be  the  result  of  an  overall  operator  information  overload,  but  this 
hypothesis  is  not  consistent  with  the  findings  of  the  questionnaire 
analyses.  If  the  augmented  feedback  created  an  information  overload, 
it  would  appear  reasonable  to  assume  that  subjects  would  ignore  the 
feedback  since  the  task  could  be  performed  without  it.  However,  of 
the  subjects  who  received  augmented  feedback,  a  significant  majority 
said  they  used  it  in  performing  the  task  (£  <  0.005).  Furthermore,  of 
the  subjects  who  used  the  feedback,  a  significant  majority  considered 
It  to  be  more  than  helpful  (£  <  0.005).  In  addition,  over  50Z  of  the 
subjects  receiving  feedback  indicated  that  they  used  it  for  at  least 
two-thirds  of  the  training  session. 

Another  interesting  point  from  the  questionnaire  regarded  the 
preference  of  feedback  type  and  actual  performance  when  presented  with 
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a  single  feedback  type.  When  subjects  who  were  presented  with  auditory 
and  visual  feedback  in  both  training  procedures  were  asked  which  feed¬ 
back  type  they  would  choose  if  they  had  to  do  the  same  task  again  (see 
question  14  in  Appendix  D) ,  a  significant  majority  chose  auditory 
feedback  (£  <  0.005).  Nevertheless,  subjects  who  received  only  the 
auditory  feedback  in  both  the  adaptive  and  fixed-difficulty  training 
procedures  did  not  have  significantly  lower  tlme-to-criterion  scores 
than  subjects  who  received  only  the  visual  feedback. 

Transfer.  As  in  the  results  of  the  training  analysis,  no 
significant  effect  due  to  feedback  was  found  in  transfer.  The  lack 
of  any  significant  effect  due  to  feedback  in  training  and/or  transfer 
was  not  expected.  Gordon  (1968),  Gordon  and  Gottlieb  (1967)  and 
Williams  and  Briggs  (1962)  have  all  shown  out-of-tolerance  augmented 
feedback  to  produce  a  performance  as  well  as  a  learning  effect  when 
used  with  open-loop  tasks  having  clearly  discernible  intrinsic  feed¬ 
back.  The  intrinsic  feedback  available  from  the  task  used  in  this 
study  was  certainly  clearly  discernible.  However,  Michelli  (1966) 
may  offer  an  explanation  for  the  differences  in  results.  In  varying 
the  amount  of  augmented  feedback  presented  and  the  discernibility  of 
the  intrinsic  feedback,  he  found  only  a  tendency  for  small  amounts  of 
information  from  augmenting  feedback  during  training  to  be  somewhat 
beneficial  in  transfer  with  tasks  having  clearly  discernible  intrinsic 
feedback. 

Although  the  augmented  feedback  in  the  fixed-difficulty  closed- 
loop  task  was  usable,  it  did  not  provide  Information  that  was  not 
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readily  available  from  the  task  itself.  This  may  help  explain  the 
nonsignificant  transfer  results.  However,  in  the  transfer  results 
of  the  adaptive  training  groups,  a  tendency  of  the  feedback  groups  to 
perform  better  in  transfer  is  evident.  The  lack  of  a  significant 
result  could  be  due  to  the  small  amount  of  additional  information 
provided  by  the  augmented  feedback  which  was  not  intrinsically  avail¬ 
able  from  the  task  (task  difficulty  information). 

Another  plausible  explanation  for  the  finding  of  a  learning 
effect  in  other  studies  employing  off-course  feedback  and  tasks  with 
clearly  discernible  intrinsic  feedback  may  be  that  of  negative 
reinforcement.  Since  the  augmenting  cues  were  only  presented  when  a 
subject's  response  was  incorrect,  an  aversive  conditioning  effect  may 
have  occurred.  Payne  and  his  associates  (Payne,  1970;  Payne  and 
Artley,  1972;  Payne  and  Dunman,  1974;  and  Payne  and  Richardson,  1972) 
have  shown  that  the  pairing  of  an  out-of-tolerance  augmenting  cue 
(red  light)  with  an  aversive  stimulus  (shock)  prior  to  performing  the 
actual  training  task  with  clearly  discernible  intrinsic  feedback  will 
cause  the  augmenting  cue  to  produce  a  performance  as  well  as  a  learning 
effect.  Groups  that  did  not  receive  the  aversive  conditioning  prior 
to  performing  the  task  failed  to  show  any  tendency  of  a  performance  or 
learning  effect  due  to  the  augmented  feedback. 

Training  Procedure 

Training.  In  training,  no  significant  effect  on  time-to-criterion 
scores  due  to  training  procedure  was  found.  This  is  consistent  with 
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the  earlier  findings  of  Cote,  et  al.  (1978).  However,  ic  was  hypo¬ 
thesized  chat  Che  training  by  feedback  interaction  would  be  significant 
due  to  the  adaptive  procedure  which  provides  tailored  training  and 
the  level-of-difficulty  feedback  which  provides  performance  infor¬ 
mation.  This  hypothesis  was  based  on  the  results  of  numerous  studies 
where  adaptive  training  has  not  been  found  to  be  superior  to  fixed- 
difficulty  training  and  Kelley's  (1969)  suggestion  that  the  failure 
of  adaptive  systems  may  sometimes  be  due  to  their  inherent  lack  of 
performance  information.  The  provision  of  level  of  difficulty  infor¬ 
mation  appears  to  be  especially  important  in  adaptive  systems  employing 
a  small-step  adaptive  logic  in  that  performance  improvement  is  then 
difficult  to  detect.  However,  the  results  of  this  study  in  which  a 
small-step  adaptive  logic  was  employed  fail  to  support  this  hypothesis. 

The  failure  of  augmented  feedback  to  improve  the  performance  of 
those  in  the  adaptive  training  procedure  may  have  been  due  to  a  couple 
of  features  inherent  to  the  task.  In  the  adaptive  system  used,  task 
difficulty  was  adjusted  in  accordance  with  the  trainee's  performance 
in  an  effort  to  keep  the  trainee's  tracking  error  relatively  constant 
at  a  specified  low  level.  As  a  consequence,  though,  behavior  producing 
out-of-tolerance  performance  one  moment  would  not  cause  out-of¬ 
tolerance  at  a  time  in  the  near  future.  Thus,  the  nature  of  the 
adaptive  system  could  have  voided  any  useful  purpose  that  the  out-of- 
tolerance  feedback  may  have  served. 

Another  reason  for  the  failure  of  augmented  feedback  to  have  an 
effect  on  the  performance  of  those  trained  adaptively  may  have  been 
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the  subjects'  knowledge  of  the  amount  of  time  they  were  to  spend  per¬ 
forming  the  task.  Subjects  knew  that  the  experiment  was  approximately 
one  hour  in  length.  As  a  result,  they  may  have  been  simply  performing 
the  task  without  giving  much  attention  to  the  information  they  were 
presented  regarding  their  performance.  Thus,  the  information  may  not 
have  been  as  important  to  them  as  it  would  be  in  a  task  that  is  open- 
ended.  In  a  task  that  is  not  readily  learned,  requiring  training  to 
take  place  over  an  extended  period,  information  indicating  actual  per¬ 
formance  level  in  relation  to  criterion  performance  level  could  play 
an  important  motivational  or  incentive  type  role.  In  an  extended 
training  session  with  a  small-step  adaptive  logic,  an  apparent  lack  of 
Improvement  could  be  extremely  discouraging  to  the  trainee  and  there¬ 
fore  impede  performance.  The  addition  of  augmented  feedback  depicting 
actual  performance  in  relation  to  criterion  performance  may  then  help 
maintain  a  steady  rate  of  improvement  in  performance. 

Transfer.  Although  a  main  effect  of  training  type  was  signifi¬ 
cant  in  transfer  with  those  who  received  the  adaptive  training  pro¬ 
cedure  tracking  more  accurately  than  those  who  received  the  fixed- 
difficulty  procedure,  the  difference  in  transfer  tracking  error  was 
minimal.  However,  this  finding  was  consistent  with  the  earlier 
findings  of  Cote,  et  al.  (1978).  As  was  hypothesized  earlier,  the 
reason  for  superior  performance  of  the  adaptive  training  group  in 
transfer  may  have  been  due  to  the  greater  amount  of  practice  afforded 
the  adaptive  group  in  keeping  the  controlled  element  within  tolerance 


V— 


m 


50 

of  Che  forcing  function  symbol  at  various  levels  of  difficulty.  This, 
of  course,  is  a  basic  feature  of  adaptive  training  systems. 

This  result  points  out  an  important  aspect  of  adaptive  systems 
that  should  not  be  ignored.  Adaptive  training  systems  have  often  been 
criticized  for  their  development  costs,  their  failure  to  reduce 
training  time,  and  their  failure  to  improve  training  performance.  How¬ 
ever,  if  performance  in  the  transfer  situation  is  better  than  it  would 
be  if  alternative  training  procedures  were  used,  then  it  may  be  wise  to 
use  an  adaptive  training  system.  Obviously,  many  factors  would  have  to 
be  considered  in  making  the  decision  of  what  type  of  a  training  system 
to  use,  but  any  particular  system  should  not  be  excluded  based  on  its 
cost  and  the  training  results  it  produces. 

Conclusion 

The  results  of  this  study  suggest  that  augmented  feedback  in  a 
closed-loop  adaptive  training  system  with  clearly  discernible  intrinsic 
feedback  does  not  enhance  performance  in  either  training  or  transfer. 
However,  the  results  obtained  may  be  due  to:  (1)  the  limited  amount  of 
information  that  may  be  provided  by  augmented  feedback  in  an  adaptive 
system  with  clearly  discernible  intrinsic  feedback,  (2)  the  feedback 
not  being  aversive,  (3)  the  length  of  the  training  period,  or  (4)  the 
features  inherent  in  an  adaptive  training  system.  Nevertheless,  it 
appears  chat  augmented  feedback  will  not  produce  a  performance  or  a 
learning  effect  when  provided  with  an  adaptive  task  of  short  duration 
Chat  has  clearly  discernible  intrinsic  feedback. 
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Although  no  effect  due  to  augmented  feedback  was  found  in  this 
study,  it  may  be  that  augmented  feedback  may  aid  performance  in  an 
adaptive  task  without  clearly  discernible  intrinsic  feedback.  One  such 
example  would  be  a  compensatory  tracking  task.  In  open-loop  motor 
skills  research,  augmented  feedback  has  been  found  to  produce  a  per¬ 
formance  as  well  as  a  learning  effect  in  tasks  that  lack  clearly 
discernible  intrinsic  feedback  (Von  Buseck,  1965;  Gibson  and  Ventola, 
1967;  and  Lintem,  1978).  However,  in  attempting  to  apply  augmented 
feedback  in  a  closed-lsop  system  lacking  clear  intrinsic  feedback, 
one  should  follow  the  suggestions  that  have  come  out  of  the  numerous 
open-loop  studies  that  have  investigated  the  effects  of  augmented 
feedback. 

Thus,  if  augmented  feedback  is  applied  in  an  adaptive  closed- 
loop  system  lacking  clearly  discernible  intrinsic  feedback,  the  feed¬ 
back  should:  (1)  not  provide  cues  which  can  be  relied  upon  for 
successful  performance  throughout  training,  (2)  signal  out-of-tolerance 
performance,  and  (3)  direct  the  attention  of  the  trainees  to  the 
results  of  their  responses  and  thus  their  errors.  It  is  hypothesized 
that  if  these  guidelines  are  followed  in  implementing  augmented  feed¬ 
back  into  a  closed- loop  adaptive  motor  skill  training  system  lacking 
clearly  discernible  intrinsic  feedback,  a  performance  as  well  as  a 
learning  effect  would  be  realized.  This  hypothesis  is  based  on  the 
results  of  Micheli's  (1966)  research  which  suggests  that  the  more 
information  provided  by  augmented  feedback,  the  greater  the  effect  on 
performance  and  learning.  However,  one  must  be  cautious  not  to  have 


such  poor  intrinsic  feedback  that  the  trainee  relies  on  the  augmented 
feedback  to  perform  the  task.  In  this  situation,  a  severe  performance 
decrement  would  occur  upon  the  withdrawal  of  the  feedback. 


SUMMARY 


A  two-dimensional  pursuit  tracking  task  was  used  to  teach  subjects 
a  complex  perceptual  motor  skill.  The  primary  emphasis  of  this 
experiment  was  to  test  the  effects  of  off-course  augmented  feedback 
on  adaptive  motor  skill  learning.  Another  aim  was  to  examine  the  com¬ 
bined  effects  of  various  augmented  feedback  types  and  training 
procedures . 

Previous  open-loop  motor  skill  research  has  shown  off-course 
augmented  feedback  to  produce  a  learning  effect  when  employed  with 
tasks  having  clearly  discernible  intrinsic  feedback.  It  is  the  general 
consensus  that  such  feedback,  if  usable  by  the  trainee,  produces  a 
learning  effect  because:  (1)  it  emphasizes  error  due  to  incorrect 
responses  early  in  training,  (2)  the  supplemental  information  provided 
by  the  augmented  feedback  is  gradually  phased  out  as  the  trainee  be¬ 
comes  more  proficient  in  performing  the  task  such  that  an  abrupt  change 
in  information  does  not  occur  when  the  augmented  feedback  is  completely 
withdrawn,  (3)  the  trainee  car  not  depend  on  the  off-course  augmented 
feedback  to  perform  the  task  throughout  training,  and  (4)  off-course 
augmented  feedback,  if  made  readily  apparent,  does  not  distract  from 
the  task  being  performed  as  does  constant  augmented  feedback  requiring 
continuous  monitoring. 

Subjects  were  taught  a  two-dimensional  pursuit  tracking  task  with 
a  fixed-difficulty  training  procedure  or  an  automatic  adaptive  training 
procedure.  Subjects  in  each  training  procedure  were  placed  in  one  of 
four  feedback  conditions.  The  four  feedback  conditions  were: 
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(1)  off-course  auditory  feedback,  (2)  off-coursa  visual  feedback,  (3) 
off-course  auditory  and  visual  feedback,  and  (4)  no  augmented  feedback. 
Subjects  in  the  adaptive  training  procedure  who  received  augmented 
feedback  were  also  given  task  difficulty  information. 

After  training  to  criterion  and  a  5-min  rest  period,  all  subjects 
were  presented  with  a  6-min  no-feedback  transfer  task  Identical  to  the 
training  task.  The  transfer  task  consisted  of  3  task  difficulty  levels. 

In  training,  no  reliable  effect  due  to  feedback  or  training 
procedure  was  found.  In  transfer,  there  was  no  reliable  effect  due 
to  training  feedback.  However,  the  effect  of  training  procedure  was 
significant  in  transfer  with  subjects  trained  adaptively  performing 
significantly  better.  This  effect  was  anticipated  on  the  basis  that 
adaptively  trained  subjects  receive  more  practice  at  various  levels  of 
task  difficulty  during  training. 

In  conclusion,  the  results  of  this  study  suggest  that  augmented 
feedback  in  a  closed-loop  adaptive  training  system  with  clearly 
discernible  intrinsic  feedback  does  not  enhance  performance.  However, 
it  is  believed  that  level  of  task  difficulty  information  may  maintain 
a  steady  Increase  in  performance  level  in  adaptive  systems  requiring 
training  to  take  place  over  an  extended  period.  Furthermore,  based  on 
the  findings  of  numerous  open-loop  motor  skill  studies  and  the  results 
of  this  research,  it  was  hypothesized  that  augmented  feedback  may  pro¬ 
duce  a  performance  as  well  as  a  learning  effect  in  adaptive  systems 
lacking  clearly  discernible  intrinsic  feedback. 
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PREDICTING  OPTIMAL  TRAINING  GROUP  ASSIGNMENT 


Beverly  H.  Willlges,  Robert  C.  Williges,  and  Ricky  E.  Savage 


Virginia  Polytechnic  Institute  and  State  University 
Blacksburg,  Virginia 


ABSTRACT 


Multiple  regression  equations  were  used  to  assign  40  students  to  fix¬ 
ed-difficulty  or  adaptive  training  based  upon  the  shorter  predicted  time- 
to-train  score.  In  addition,  40  students  were  randomly  assigned  to  the  two 
training  conditions,  and  40  students  were  mismatched  to  training  based  upon 
the  longer  predicted  time  to  train.  Using  predicted  scores  to  match  stu¬ 
dents  to  training  alternatives  resulted  in  a  47%  savings  in  training  time 
over  random  assignment  and  a  53%  savings  over  mismatched  assignment.  The 
assignment  effect  was  reliable  at  the  0.0001  level.  Future  research  will 
examine  different  categories  of  predictors,  additional  training  alterna¬ 
tives,  and  more  complex  training  tasks. 


INTRODUCTION  maximize  student  achievement  by  adjust¬ 

ing  the  instructional  environment  to 
individual,  and  perhaps  changing, 
The  vast  storage  capacity  of  the  instructional  needs, 

computer  has  encouraged  training  desig¬ 
ners  to  provide  adaptive  instruction 

tailored  to  individual  student  needs.  A  second  approach  to  individual 

In  training,  the  role  of  the  computer  differences  in  training  involves  the  use 

varies  from  disseminator  of  information  of  the  computer  as  a  manager  of  instruc¬ 
ts'  manager  of  instruction.  As  a  disse-  tion.  Carroll’s  (1963)  learning  theory 

minator  of  information  the  computer  det-  suggests  that  the  degree  of  learning  a 

ermines  the  student's  unique  trajectory  given  task  is  a  function  of  the  amount 

through  the  curriculum.  This  is  what  of  time  spent  learning  the  task  in  rela- 

Hansen  (  1  973)  refers  to  as  within  task  tion  to  the  amount  of  time  needed  to 

adaption.  Hansen  defines  pretask  adap-  learn  the  task.  Time  needed  is  based 

tion  as  the  use  of  the  computer  as  a  upon  learning  under  optimal  conditions, 

manager  to  diagnose  and  prescribe  Bloom  (1976)  provides  three  predictors 

instruction.  of  time  to  learn:  (1)  cognitive  entry 

behaviors  (prior  experience  with  the 
task),  (2)  affective  entry  behaviors 
Much  research  has  been  directed  (motivation  level  of  the  student),  and 

toward  the  development  of  an  optiroiza-  (35  quality  of  instruction  ( appropr isle¬ 
ts  jn  model  to  disseminate  information.  ness  for  the  student). 

Fletcher  (1975)  describes  four  types  of 
quantitative  models:  memory  (e.g., 

Estes,  I960),  artificial  intelligence  As  a  manager  of  instruction  the 

le.g.,  Newell,  Shaw,  and  Simon,  I960),  computer  must  use  a  model  to  determine  a 

automation  (e.g.,  Minsky,  1967),  and  priori  the  appropriateness  of  a  particu- 

regression  (e.g.,  Suppes,  Fletcher,  and  lar  training  method.  For  example, 

Zanotti,  1975).  An  earlier  approach  to  regression  models  might  be  useful  to 

instructional  sequencing,  Smallwood's  predict  training  success.  Kaskowitz  and 

(i)62)  model,  uses  both  the  student's  Suppes  (1978)  have  suggested  that  a 

response  history  as  well  as  the  cumula-  regression  equation  may  be  considered  to 

live  history  of  students  who  took  previ-  be  a  mathematical  model  in  the  sense 

ous  courses.  In  motor  skill  training  that  a  linear  relationship  between  rate 

the  linear  adaptive  model  developed  by  of  learning  and  certain  independent  var- 

Kolley  (1969)  has  been  the  stimulus  for  iables  is  hypothesized, 

a  great  deal  of  research.  The  goal  of 
each  of  these  approaches  is  to  minimize 

time  ind  instructional  costs  and  to  Two  findings  reported  by  Wagner, 

Coov right  ©  1979,  The  Human  Factors  Society,  Inc.  All  rights  reserved. 
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Behringer,  and  Pattie  (1973)  provided 
impetus  for  the  present  study  in  which 
regression  equations  were  used  to  pred¬ 
ict  optimal  training  group  assignment. 
Using  regression  equations  to  predict 
time  to  complete  a  course  on  stock  con¬ 
trol  and  accounting,  Wagner  et  al .  found 
that  (1)  simple  mathematical  equations 
were  the  best  predictors  of  performance 
and  (2)  grouping  students  according  to 
mode  of  instruction  (audio-visual  or 
programmed  instructions)  improved  pred¬ 
iction.  The  improvement  in  prediction 
wnen  students  are  grouped  by  training 
type  suggests  that  training  type  inter¬ 
acted  with  individual  differences  among 
students.  Perhaps  the  best  score  pred¬ 
icted  with  the  various  training  types 
could  be  used  to  select  an  optimal 
training  assignment. 


A  preliminary  evaluation  of  the  use 
of  multiple  regression  for  training 
group  assignment  has  been  conducted 
using  the  Air  Force  Advanced  Instruc¬ 
tional  System's  Inventory  Management 
course.  McCombs  (1979)  reports  modest 
savings  in  training  time  when  regression 
models  were  used  to  select  students  for 
alternative  training  modules.  However, 
because  the  study  was  conducted  within 
the  constraints  of  an  operational  train¬ 
ing  system,  several  limitations  should 
be  noted.  First,  alternative  treatments 
were  available  only  in  selected  lessons 
(27*  of  the  course).  Second,  no  stu¬ 
dents  were  purposefully  mismatched,  so 
tne  d iscriminabil ity  of  the  selection 
procedure  could  not  be  tested.  Third, 
selection  of  the  optimal  training  type 
could  be  overridden  when  tne  instruc¬ 
tional  materials  were  not  available  or 
when  an  instructor  changed  a  student's 
assignment.  However,  even  with  these 
limitations,  consistent  savings  in  time 
to  learn  a  cognitive  task  were  obtained 
using  regression  modeling. 


The  present  study  extended  this 
regression  model  approach  to  the  percep¬ 
tual-motor  learning  domain.  Specifi¬ 
cally,  groups  were  assigned  to  various 
training  strategies  on  the  basis  of 
predicted  scores  from  baseline  regres¬ 
sion  models  of  training  time-to-exit. 
Students  were  matched  (shorter  predicted 
time),  mismatched  (longer  predicted 
time),  or  randomly  assigned  to  fixed- 
difficulty  or  adaptive  training  to  learn 
a  two-dimensional  pursuit  tracking  task. 


METHOD 


Regression  Equations 


A  battery  of  six  tests  and  sex  T 
the  student  were  used  to  provide  predic¬ 
tor  variables.  The  pretest  battc-ry 
included:  (1)  pursuit  rotor  (motor 

skill),  (2)  Embedded  Figures  Test  (field 
independence),  (3)  Identical  Pictures 
Test  (perceptual  speed),  (4)  Mate  Trac¬ 
ing  Test  (spatial  scanning),  (5)  Map 
Memory  Test  (visual  memory),  and  (6) 
Cube  Comparison  Test  (spatial  orienta¬ 
tion).  The  Embedded  Figures  Test  is 
from  the  Educational  Testing  Service 
(Witkin,  Oltman,  Raskin,  and  Karp, 
1971),  and  the  last  four  tests  are 
paper-and-penc i  1  tests  from  the  Ekstrom, 
French,  Harman,  and  Derman  (1976)  bat¬ 
tery.  .  Five  stepwise  regression  proce¬ 
dures  were  used  to  determine  equations; 
those  with  the  fewest  predictors  and 
accounting  for  the  most  variance  were 
selected  . 


A  double  cross-validation  procedure 
was  U3ed  to  validate  the  regression 
equations  which  predicted  time  L.<  lo  in 
a  two-dimensional  pursuit  tracking  task. 
The  coefficients  of  multiple  determi-:,- 
tion  were  consistently  high,  so  the  two 
samples  were  combined  and  new  equations 
generated.  The  combined  sample  equa¬ 
tions,  given  in  Table  1,  were  used  for 
training  group  assignment. 


Experimental  Design 


A  2  x  2  x  3  complete  factorial 
design  with  two  levels  of  sex,  two  lev¬ 
els  of  training  (fixed-difficulty,  adap¬ 
tive),  and  three  levels  of  assignment 
(matched,  random,  mismatched)  was  used. 
A  total  of  120  undergraduates  were 
equally  divided  among  the  12  cells  of 
the  experiment.  All  subjects  were 
volunteers  and  were  paid  for  their  par¬ 
ticipation. 


Equipment  and 


Tasks 


A  PDF  11/55  digital  computer  pro¬ 
vided  inputs  to  a  Tektronix  4  0 1  a  - 1 
cathode  ray  tube  display  and  processed 
control  inputs  from  a  Measurement  System 
Model  <*35  isometric  control  stick. 
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Each  subject  completed  a  series  of 
trials  to  learn  a  two-dimensional  track 
ing  task  in  which  random  functions  were 
used  to  determine  the  x-y  coordinates  of 
the  forcing  function  symbol  (X). 

Table  1 


difficulty  changed  automatically  after 
each  minute  of  tracking.  Three  levels 
of  difficulty  were  used:  the  same  as 
exit  criterion  in  training-,  more  diffi¬ 
cult  than  exit  criterion,  and  less  dif¬ 
ficult  than  exit  criterion. 


f'cnbined  Sample  Regression  Equations  for  Training  RESULTS  AND  DISCUSSION 

Tirne-tO-Exi  t 


Adaptive  Training 

ft  *  1326.85  +  381.82  EF  -  307.48  m  +  259.52  SE 

n  *  51 
R~  *  .756 
R|  *  . 740 
£  <_  »  .  0001 

FI  \eii-D i  I  I  I  c u_l  tv  Training 

I  I  '>•!.•,.  5 7  I  405.  77  KF  +  271.10  IP  -  119.28  CC 

n  =*  48 
P.2  *  .612 

4.2  >  .607 

£  £  »  . 000 1 


FF  =  Embedded  Figures  Test 

'17!  -  Map  Memory  Test 

8!"  *  Sex  of  Student 

IP  -  I  den  lira  I  Pictures  Test 

'T  .  Cube  Comparison  Test 

Tli  s  control  output  (0)  was  generated 
using  inputs  from  the  analog  controller. 
Dynamic,  augmented  visual  feedback  in 
terms  of  tracking  error  and  task  diffi¬ 
culty  was  provided.  Task  difficulty  was 
manipulated  in  terms  of  the  movement 
speed  and  distance  of  the  forcing  func¬ 
tion  symbol.  Exit  criterion  was 
obtained  when  the  student  maintained 
exit  criterion  task  difficulty  and 
acceptable  accuracy  for  a  period  of  20 
seconds . 


Training  T ime 


The  correlation  between  predicted 
and  actual  time-to-exit  scores  was  .757. 
For  only  the  students  receiving  adaptive 
training  the  correlation  was  .789;  for 
fixed-difficulty  training  it  was  .716. 
Coefficients  of  multiple  determination 
were  somewhat  lower  than  the  estimates 
of  shrinkage  (R  d)  given  in  Table  1. 


Results  of  an  analysis  of  variance 
on  actual  training  time-to-exit  score 
revealed  reliable  main  effects  o. 
assignment  F(2 , 1 08) s 1 7 . 27 , p< . 000 1 ,  and 
sex,  F  ( 1 ,  1 0£D  :40 . 57  ,  p<  .  00*01.  Matched 
subjects  required  significantly  less 
time  to  exit  than  eitner  random  or  mis¬ 
matched  subjects,  and  males  required 
significantly  less  training  than 
females.  There  was  no  reliable  differ¬ 
ence  between  training  alternatives 
(£s.2«6> . 


Use  of  the  regression  equations  to 
predict  optimal  training  type  resulted 
in  savings  of  47%  of  training  time  over 
random  assignment  and  53%  over  mis¬ 
matched  assignment.  Variance  in  train¬ 
ing  time  was  reduced  approximately  40% 
by  optimizing  training  group  assignment. 
Table  2  summarizes  the  reliable  effects 
from  the  analysis  of  training  time. 


Two  training  conditions  were  avail- 
able--f ixed-d i f f icul ty  and  adaptive.  In 
fixed-difficulty  training  the  criterion 
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as  a 
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culty  is  presented 
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Adaptive  training 
closed-loop  system 
iulty  varies  during 
on  of  student  per- 


f  ormance . 


Following  training  and  a  short  rest 
period,  each  subject  completed  a 
7 -minute  transf.r  task  in  which  no  aug¬ 
mented  feedback  was  provided.  Task 


These  data  strongly  support  the  use 
of  regression  equations  to  optimize 
training  group  assignment.  The  optimi¬ 
zation  procedure  resulted  in  savings  in 
training  time  and  a  reduction  in  vari¬ 
ance  among  students.  Interestingly,  no 
overall  difference  in  training  time  bet¬ 
ween  fixed-difficulty  and  adaptive 
training  was  noted.  If  the  study  hod 
employed  only  random  assignment,  one 
might  have  erroneously  concluded  that  no 
advantage  is  to  be  gained  in  providing 
alternative  training  conditions. 
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Transfer  Task  Accuracy 


CONCLUSIONS 


An  analysis  of  variance  on  vector 
root  mean  square  tracking  error  in  the 
transfer  task  revealed  a  reliable  mai 
effect  of  level  of  difficulty 

K  (2  ,  2  1 6 )  s  386 . 2 ,  gt.OOOl,  indicating  that 
the  three  levels  of  tracking  difficulty 
did  represent  different  skill  levels  for 
the  students. 


The  performance  differences  between 
males  and  females  also  appeared  in 
t  rjnsfer  with  males  performing  more 
accurately  than  females,  F ( 1 , 108) s 1 9 . 53, 
p;. 0002.  In  addition,  there  was  3  reli¬ 
able  interaction  between  level  of  diffi¬ 
culty  and  sex,  F (2 , 2 1 6) r5 . 30 ,  p=. 00057. 
Although  all  post  hoc  comparisons  were 
reliable  (g<.05),  the  superiority  of 
males  over  females  was  most  pronounced 
at  the  highest  level  of  task  difficulty. 


The  main  effect  of  assignment  was 
again  reliable  in  transfer,  F(2,108)  = 

7.98,  g=.0006.  Post  hoc  analyses  corn- 

firm  that  the  reliable  differences  in 
transfer  were  between  random  and  mis¬ 
matched  assignment  and  between  matched 
and  mismatched  assignment.  It  is  impor¬ 
tant  to  point  out  that  the  assignment 
optimisation  procedure  was  based  on 
training  time,  not  transfer  performance. 
Therefore,  marked  differences  m  trans¬ 
fer  were  not  expected  since  students 
were  trained  to  the  same  exit  criterion. 


•  il.le  2 


uinmary  Statistics  for  Training  Tlme-to-Exlt  (min' 


1  raining  Type 


Adaptive 

1 -.5 

i:.a 

Fixed-Dlf ficultv 

15. 6. 

10.1 

Assignment  Procedure 

Matched 

10.0 

7.0 

Random 

18.' 

11.6 

Mismatched 

21.  1 

11.: 

RcX  of  Scudenc 

Ma  1  e 

11. A 

■ .  1 

Femi  le 

21.8 

12.  A 

These  data  clearly  support  the 
notion  that  training  con  be  improved  bv 
providing  alternative  trainin';  pro.-.  - 
dures  with  assignment  baaed  upon  hi 
optimization  model.  In  addition,  too 
research  supports  the  cffie.icy  of  1 
regression  approach  for  iss  ignmeui 
optimization. 


To  facilitate  implementation  m 
operational  training  systems,  research 
is  warranted  to  examine  regress  1  in 
optimization  with  additional  types  f 
predictors,  training  procedures,  -:U 
more  complex  training  tasks.  Guidelines 
for  selection  of  viable  predictors  is 
particularly  critical.  The  experience 
with  the  Advanced  Instructional  System 
vividly  portrays  the  complexity  of 
implementing  innovative  training  pro¬ 
grams.  However,  if  the  computer  is  evc- 
to  provide  real  value  to  trainin'  sys¬ 
tems,  the  challenge  of  the  opor-.tiji.il 
training  system  must  be  conquered. 
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Appendix  D 

Prediction  of  Performance  in  Motor  Skills  Training 
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UNCLASSIFIED  VPI-HFL-79-11/AF0SR-79-1  AFOSR-TR-80-02B4  NL 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  OF  STANDARDS- 1963-A 
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PREDICTION  OF  PERFORMANCE  IN  MOTOR  SKILLS  TRAINING 
Richard  J.  Becker,  Beverly  H.  Utilises,  and  Robert  C.  Utilises 

Virginia  Polytechnic  Institute  and  State  University 
Blacksburg,  Virginia 

and 

Jefferson  M.  Koonce 
USAF  Academy,  Colorado 

ABSTRACT 

Willlges,  Wllllges,  and  Savage  (1979)  demonstrated  the  ucllley  of  a  multiple-regression 
approach  for  assigning  individuals  to  training  alternatives.  Average  training  time  savings 
of  50Z  were  obtained  when  the  lower  predicted  score  was  used  to  assign  students  to  fixed- 
difficulty  or  adaptive  training  conditions  to  learn  a  two-dimensional  tracking  task.  The 
present  study  extends  this  work  by  developing  prediction  models  for  a  real-world  training 
task.  Regression  models  were  developed  to  predict  performance  on  several  flight  manuevers 
using  the  ATC-610  Flight  Trainer.  One  hundred  VPI  undergraduates  and  100  USAF  cadets, 
with  an  equal  number  of  males  and  females,  served  as  subjects. 

The  predictors  included  measures  of  information  processing  skills,  motor  skills,  and 
demographic  characteristics.  The  measures  of  Information  processing  skills  were:  O) 
Embedded  Figures  Test  (field  Independence);  (2)  Identical  Pictures  Test  (perceptual  speed); 

(  I)  Maze  Tracing  Test  (spatial  scanning);  (4)  Map  Memory  Test  (visual  memory);  and  (5) 

Cube  Comparisons  Test  (spatial  orientation) .  The  motor  skills  tests  were  the  Psychomotor 
Test  Device  Tests  1  and  2  (systems  Research  Laboratories)  and  the  pursuit  rotor.  Demographic 
predictors  included  sex  and  educational  institution.  Difference;  in  reliable  predictors 
for  males  versus  females,  cadets  versus  undergraduates,  and  tracking  versus  flight  trainer 
performances  are  discussed. 


Copyright  Q  1979,  The  Human  Factors  Society,  Inc.  All  rights  reserved. 
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